Kubernetes has become the de facto operating system for AI. In CNCF’s 2025 Annual Cloud Native Survey, 82% of container users now run Kubernetes in production, and 66% of organizations hosting generative AI use it to manage some or all of their inference workloads. But few teams run AI in continuous production yet: only 7% of organizations deploy models daily, and 47% only intermittently.
That gap is operational, and it widens as AI broadens from training into inference, fine-tuning, and agentic workloads that have to run reliably and continuously. Running those at scale is a distributed systems problem, and distributed systems are what the cloud native community has spent a decade perfecting how to run at scale. The opportunity now is to make GPU workloads as routine on Kubernetes as any other workload.
That is why NVIDIA believes the future of AI will be built in the open, and why we are investing in the community building it.
Kubernetes is already the foundation for AI
For more than a decade, the cloud native community has built the infrastructure that runs the modern internet. Kubernetes and the CNCF ecosystem gave organizations a common way to orchestrate complex, distributed applications across heterogeneous environments at global scale. AI workloads are landing on that same foundation because they need the same things: dynamic scheduling, observability, networking, and coordination that holds together when individual components fail.
What is still missing is first-class support for GPUs in that foundation. The cloud native ecosystem knows how to schedule, network, observe, and secure distributed workloads. Bringing that same flexibility to GPUs is the opportunity in front of the community.
What the community has been telling us
The maintainers and platform teams we work with have told us the same thing consistently: operating GPUs efficiently at scale requires deeper collaboration, because the current tools were not designed for it. Historically, Kubernetes treated GPUs as static, indivisible resources. That was fine for early workloads. It breaks down the moment an organization runs large-scale training, latency-sensitive inference, and multi-tenant AI platforms on shared clusters, which is now the common case. The result is the familiar tax: overprovisioned hardware, accelerators sitting idle, and applications that pass every test and then fall over in production. That tax is one reason so many teams still deploy models only occasionally rather than continuously.
This is an orchestration problem, and orchestration problems are best solved in the open, as shared standards rather than vendor-specific extensions bolted on around the edges. That conviction is what is changing how NVIDIA shows up in this community.
Deepening our commitment to CNCF
NVIDIA already builds and maintains open source that the cloud native ecosystem runs on, including the NVIDIA Container Toolkit and the GPU Operator. What is changing is the depth of the commitment. This year we joined the CNCF Governing Board and committed $4 million over the next three years so CNCF projects can run their CI and testing on real GPUs instead of emulators. The way we work is consistent: contribute upstream first, share governance rather than control it, and stay with the projects we help start. Recent work shows what that looks like.
The NVIDIA GPU Dynamic Resource Allocation (DRA) Driver is now upstream in Kubernetes SIG-Node, as the reference implementation for the vendor-neutral DRA API. It replaces static assignment at scheduling time with real-time, on-demand GPU allocation, adds MIG device sharing (in alpha), and introduces ComputeDomains, which let GPUs share memory safely and quickly across nodes over Multi-Node NVLink. In plain terms, it lets a large multi-GPU job get exactly the accelerators it needs, when it needs them, instead of holding a static reservation. We contributed it upstream under community governance rather than shipping it as a proprietary extension because GPU management belongs in the Kubernetes standard.
The KAI Scheduler was accepted as a CNCF Sandbox project at KubeCon + CloudNativeCon Europe. KAI handles the scheduling that large AI clusters actually demand: gang scheduling with pre-scheduling simulation so jobs are not evicted needlessly, hierarchical queues with Dominant Resource Fairness for multi-team clusters, and asynchronous binding that has been run against clusters of more than 10,000 GPUs. Bringing it into the Sandbox puts its roadmap under community direction rather than a product timeline. It is the same engine NVIDIA relies on internally, and we would rather the whole ecosystem build on it and shape it than keep it to ourselves.
Verification is the third piece. The Kubernetes AI Conformance Program has grown from 18 to 31 certified platforms since it launched a few months ago, giving the ecosystem a way to confirm that AI-ready infrastructure works consistently across providers rather than taking each vendor’s word for it. New requirements landing in v1.35 cover agentic workflow support and in-place pod resizing for inference serving.
The principle under all of it is simple: the foundational infrastructure for running AI should be a community asset.
An open future, by choice
The future of AI infrastructure is far from written. It could consolidate around a handful of proprietary, closed platforms. Or it could follow the path the internet took, with Linux and Kubernetes and countless open foundations that produced more global innovation than any single company could have on its own. NVIDIA is betting on the second path, not out of ideology but because the historical record on infrastructure is one-sided: open, interoperable, community-governed platforms build larger ecosystems, move faster, and benefit more people than closed ones do.
As Jensen Huang put it: “We want AI to diffuse into every industry and every country, every researcher, every student. And if everything is proprietary, it’s hard to do research and it’s hard to innovate on top. And so open source is fundamentally necessary.”
That is the conviction behind NVIDIA’s commitment to CNCF. The future of AI will not be built by any one company. If you work in SIG-Node, on DRA, on the AI Conformance Program, or anywhere in AI scheduling and inference, the roadmap is public and the work is upstream. Come build it with us.