GPU sharing in Kubernetes is often framed as a cost problem, since small models can leave a whole card mostly idle. Here is how the main options differ.
The device plugin exposes each NVIDIA GPU to the scheduler as a whole-number count, nvidia.com/gpu, which is why sharing methods have to work around that limit. Time-slicing advertises several replicas of one GPU but isolates nothing: every workload can reach all of the memory, and a crash in one affects them all. MIG, available on Ampere and newer data center cards such as the H100, splits a GPU into up to seven instances with separate memory paths and fault isolation, though reconfiguring it disrupts running workloads and idle slices cannot lend capacity to each other. MPS shares memory and compute and lets kernels from different processes run concurrently, but a fatal fault can reach every client on the GPU, and the device plugin labels its support experimental.
Dynamic Resource Allocation takes a different approach. Instead of counting devices, pods reference claims that select hardware by attributes such as memory capacity, using CEL expressions. DRA reached general availability in Kubernetes 1.34, but several NVIDIA driver features remain alpha, and the scheduler does not preempt for DRA resources yet. Migration from the device plugin also requires a new cluster when the GPU Operator is used.
Measuring whether sharing actually saves money matters as much as the setup. A pod that requests a GPU holds it whether or not it is working, so teams should compare real utilization from the DCGM exporter against allocation. Sharing only pays off if the cost per request or per training run falls while latency targets still hold.
For most teams, the choice depends on who shares the card: MIG suits workloads that must not affect each other, time-slicing suits trusted development and bursty jobs, and large training runs are best kept on whole GPUs.
Source: Dev.to
