
Why AI infrastructure needs one operating system, not three more platforms to manage
Every generation of infrastructure promises to clean up the mess left by the last one, and every generation ends up creating a bigger mess of its own. Virtual machines were supposed to tame physical server sprawl. Kubernetes was supposed to tame VM sprawl. And now AI agents—agent sprawl infrastructure at scale—spun up by the thousands to reason, retrieve, and act on our behalf, are doing to Kubernetes what Kubernetes did to VMware. The pattern is not new. Only the scale is.
The original sin: VM sprawl
Back in the mid-2000s, virtualization felt like magic. A single physical server that used to run one application, often at a fraction of its capacity, could suddenly host a dozen isolated virtual machines. IT teams provisioned VMs the way developers today spin up containers: quickly, cheaply, and with very little friction. That ease of creation is exactly what caused the problem.
Within a few years, data centers were full of forgotten VMs still consuming CPU, memory, and storage long after the project that justified them had ended. The industry needed a word for it, and “VM sprawl” stuck. The term is often associated with the VMware era of the early 2010s, a period when Pat Gelsinger, who took over as VMware’s CEO in 2012, was one of the most visible voices describing what unchecked virtualization was doing to enterprise data centers. Exactly who said it first is harder to pin down than the industry folklore suggests, but the diagnosis was accurate regardless of who wrote the label on the bottle: uncontrolled provisioning, orphaned snapshots, oversized allocations, and storage capacity quietly bleeding away.

Illustrative, not measured: each new abstraction layer has historically multiplied the number of things an operator has to track, not reduced it.
Kubernetes: ten times the abstraction, ten times the sprawl
Kubernetes arrived with a genuinely good idea: stop managing individual machines and start managing declarative, self-healing workloads. But the same ease-of-creation problem that plagued VMs came back wearing a different badge. Where a VM sprawl incident might have meant a few hundred forgotten virtual machines, a Kubernetes cluster multiplies the unit of concern into pods, containers, sidecars, namespaces, persistent volume claims, config maps, and custom resources, often across dozens of clusters spanning multiple clouds.
A single microservice can now spawn a small constellation of objects, each with its own lifecycle, each capable of being forgotten. Storage got hit especially hard. Every namespace wants its own persistent volumes, every team wants its own storage class, and every cluster tends to accumulate its own island of capacity that nobody fully owns. What used to be VM sprawl on a handful of hypervisors became container sprawl across fleets of clusters, and the operational burden of tracking it all grew roughly by an order of magnitude. The tools got more powerful. The mess got worse.
Storage teams felt this most acutely. A VM sprawl problem could usually be solved by an administrator running a report and reclaiming a few terabytes. A Kubernetes sprawl problem meant chasing persistent volumes across clusters that different teams owned, often with no single source of truth for what was actually running where, or why. The common thread across both eras is that nobody designed the data layer to be the one place that saw everything. Compute got an orchestrator. Storage got left to catch up.

Three sprawl cycles, twenty years apart, each one an order of magnitude bigger than the last.
AI agents: sprawl with a mind of its own
If Kubernetes multiplied sprawl by ten, autonomous AI agents are multiplying it again, and this time the sprawl is not even fully under human control. Agents are ephemeral by design. They get spun up to answer a question, call a tool, retrieve a document, write to a database, or hand off to another agent, and then they vanish, sometimes leaving behind logs, embeddings, cached context, and partial state that nobody explicitly asked to keep.
Multiply that by the number of agents a modern enterprise is starting to run, from customer service bots to coding assistants to autonomous data pipelines, and the sprawl stops being a storage housekeeping problem and becomes an existential data management problem. Every agent needs fast access to the same underlying data. Every agent generates its own trace of activity. Every agent call is, in effect, a tiny, transient workload with its own resource footprint, and there can be thousands of them running per second. VM sprawl was measured in machines. Kubernetes sprawl was measured in objects. Agent sprawl is measured in events, and events do not sit still waiting for someone to clean them up.
Don’t feed it after midnight, don’t get it wet
There is an old bit of movie folklore about small, harmless-looking creatures that multiply out of control the moment you break two simple rules: don’t feed them after midnight, and never, ever get them wet. It is a silly premise, but it is also an uncomfortably good metaphor for infrastructure sprawl. Every unmanaged copy of data, every orphaned volume, every forgotten agent process is that same creature. Feed it more compute after hours with nobody watching, and it multiplies. Add water, meaning a flood of new demand from an AI initiative that nobody planned the data layer for, and it multiplies again, faster than anyone expected, until the mess is running the show instead of the other way around.

The lesson generalizes past any one movie: uncontrolled provisioning plus a sudden flood of new demand is how sprawl gets out of hand.
The rule enterprises have learned the hard way, three sprawl cycles in, is that you cannot fix a multiplying problem by adding another disconnected platform on top of it. Bolting a new orchestration layer onto an already siloed storage estate just gives the sprawl a new place to hide.
Tying it together: VAST AI OS 5.5 and AgentEngine
This is precisely the problem VAST Data built its AI Operating System to solve, and the 5.5 release sharpens that focus. Instead of treating agents as yet another workload that needs its own storage silo, its own database, and its own bolted-on pipeline, VAST AI OS runs everything on a single Disaggregated Shared-Everything architecture. One global namespace. One data layer for files, objects, tables, and vectors. No separate systems to reconcile, and no separate copies quietly multiplying in the dark.

Before: a new silo for every layer of the stack. After: files, objects, tables, vectors, and agents all sitting on one DASE-backed namespace.
At the top of that stack sits AgentEngine, VAST’s auto-scaling runtime for deploying and operating AI agents. Rather than letting every team spin up its own ad hoc agent infrastructure, complete with its own vector store, its own event bus, and its own forgotten leftovers, AgentEngine gives agents a shared, governed environment: a low-code way to select reasoning models, define the tools an agent can call, and operationalize reasoning at scale, all directly against the same DASE-backed data layer that everything else in the AI OS already uses. An agent tool server lets those agents invoke data, metadata, functions, web search, and even other agents through MCP-compatible tools, which means the integration surface stays small and shared instead of multiplying with every new use case.
That matters for sprawl in a very concrete way. Because agents in AgentEngine share the same kernel, the same eventing and messaging infrastructure, and the same underlying DataStore, DataBase, and DataSpace services, an organization is not accumulating a new island of infrastructure every time it stands up a new agent. Observability comes along for the ride too, with distributed tracing across agentic pipelines, so the thousands of ephemeral agent events that would otherwise be invisible become something an operator can actually see and govern, instead of discovering six months later as an unexplained bill or an unexplained outage.

The VAST AI Operating System stack: DASE as the foundation, a kernel to run services across clouds, core data and messaging services, and AgentEngine as the runtime for agentic workflows.
In other words, AI OS 5.5 is not trying to out-orchestrate the sprawl with yet more orchestration. It is trying to remove the reason the sprawl exists in the first place, which is that every layer before it gave teams an easy way to create something new and no easy way to see, govern, or reclaim it later. One architecture, one namespace, one place to look.
The pattern, once more, with feeling
VM sprawl taught the industry that ease of creation without visibility is a liability. Kubernetes proved the lesson didn’t stick, and multiplied it by an order of magnitude. AI agents are proving it again, faster and with less human oversight than either of the two waves before them. The organizations that get ahead of this round will be the ones that stop adding platforms on top of the mess and start consolidating onto a data layer built to hold all of it, files, tables, vectors, and now agents, without spawning a new silo every time someone asks for one more capability.
Don’t feed it after midnight. Don’t get it wet. Or better yet, build the one system that was designed to handle the water in the first place.
Related on Lots of Data: Agent Sandboxes Storage Trends · K8s Remote Storage for AI Part 1 · VAST AI OS API Documentation Hub
