MIG vs Time-Slicing vs MPS: GPU Architecture Compared for AI

As enterprise artificial intelligence workloads diversify across simultaneous inference requests, embedding generation, and fine-tuning jobs, maximizing the return on investment of modern GPU accelerators has become a primary engineering priority. An NVIDIA H100 SXM5 accelerator with 80 GB of High Bandwidth Memory (HBM3) and 3.35 TB/s of memory bandwidth represents immense computational capacity. For lightweight inference microservices or intermittent batch tasks, dedicating an entire 8-way GPU node to a single process can result in poor hardware utilization and inflated operating costs.

To optimize resource efficiency, infrastructure architects evaluate three distinct GPU sharing mechanisms: Time-Slicing, CUDA Multi-Process Service (MPS), and NVIDIA Multi-Instance GPU (MIG). However, each approach embodies fundamental trade-offs between hardware isolation, memory bandwidth predictability, context-switching overhead, and fault tolerance. In high-concurrency enterprise production, misconfiguring these technologies introduces severe tail latency spikes and system-wide memory crashes. This technical guide delivers an in-depth architectural comparison of MIG, Time-Slicing, and MPS, mathematical formulations of memory contention, production configuration scripts, and evaluation criteria for bare-metal isolation.

Detailed Architecture of GPU Sharing Mechanisms

Understanding the operational trade-offs requires examining how each technology interacts with the underlying silicon, compute cores, and memory controllers.

1. Time-Slicing (Temporal Sharing)

Time-slicing is the default sharing mechanism utilized by the standard NVIDIA Kubernetes Device Plugin. Multiple containers are assigned to the same physical GPU, and the hardware driver rotates access between them using round-robin time quantums. Crucially, time-slicing provides zero memory isolation. All co-located containers share the same global VRAM address space; if one workload experiences an unexpected memory allocation spike, it triggers an unrecoverable CUDA Out of Memory (OOM) error that terminates adjacent containers. Furthermore, constant context swapping flushes L2 cache lines, causing severe tail latency degradation.

2. CUDA Multi-Process Service (MPS)

CUDA MPS addresses temporal inefficiency by multiplexing multiple host processes into a single unified CUDA context on the GPU. Instead of taking turns sequentially, multiple client applications execute kernels concurrently across available Streaming Multiprocessors (SMs). MPS allows administrators to configure thread percentage limits per client (e.g., allocating 40% of SMs to Client A and 60% to Client B). While MPS maximizes SM utilization and eliminates context-switching latency, it lacks hardware-level memory bandwidth isolation. A memory-intensive kernel executed by one client can saturate the memory bus, starving neighboring processes.

3. NVIDIA Multi-Instance GPU (MIG)

Introduced on Ampere and refined on Hopper architectures, MIG provides true hardware-level partitioning. An H100 GPU can be partitioned into up to seven independent GPU instances. Unlike MPS or time-slicing, MIG physically partitions the silicon: each instance receives dedicated Streaming Multiprocessors, isolated memory crossbar paths, dedicated L2 cache slices, and physically allocated HBM3 memory controllers. If one MIG instance crashes or attempts an out-of-bounds memory write, neighboring instances remain completely unaffected, providing true multi-tenant QoS guarantees.

Mathematical Model: Memory Bandwidth Contention and Tail Latency

In shared GPU architectures, memory bus contention represents the primary bottleneck governing inference throughput and tail latency.

The effective memory bandwidth $B_{\text{effective}, i}$ available to concurrent workload slice $i$ is formulated as:

$$ B_{\text{effective}, i} = \min \left( B_{\text{slice}, i}, B_{\text{bus}} \times \frac{w_i}{\sum_k w_k} \right) - \delta_{\text{context\_switch}} $$

Where $B_{\text{bus}}$ is the physical memory bus bandwidth (3.35 TB/s on H100), $w_i$ represents the memory bus arbitration weight of client $i$, and $\delta_{\text{context\_switch}}$ is the latency penalty incurred by cache invalidation and page table flushes.

In temporal time-slicing environments, the 99th-percentile inference latency ($L_{99}$) compounds rapidly under traffic concurrency:

$$ L_{99} = L_{\text{compute}} + \sum_{j \neq i} T_{\text{quantum}, j} + \Delta t_{\text{PCIe\_flush}} $$

When multiple concurrent models contend for execution, $L_{99}$ latency can spike by 4x to 10x relative to baseline execution. In contrast, NVIDIA MIG partitions eliminate $\delta_{\text{context\_switch}}$ and establish a deterministic lower bound on bandwidth $B_{\text{slice}, i}$, guaranteeing predictable latency profiles.

GPU Sharing Technologies Comparison Matrix

The technical trade-offs across virtualization and partitioning strategies are evaluated below.

While MIG and MPS excel at optimizing resource density for smaller inference models, distributed foundation model training and high-throughput inference require unconstrained access to high-speed NVLink interconnects. Because MIG disables cross-instance NVLink communication, enterprise workloads scaling across multiple GPUs achieve optimal price-performance on OneSource Cloud dedicated bare-metal infrastructure.

Production Verification: NVIDIA MIG Partitioning CLI Workflow

To configure and verify hardware-isolated MIG slices on an NVIDIA H100 system, infrastructure engineers execute low-level driver commands.

// 1. Enable MIG mode on GPU device 0 (requires root privileges)
nvidia-smi -i 0 -mig 1
// 2. Query available GPU instance profiles on H100 SXM5
nvidia-smi mig -lgip
// 3. Create two 3g.40gb hardware instances (each providing 3 SM slices and 40GB HBM3)
nvidia-smi mig -cgi 3g.40gb,3g.40gb -C
// 4. Verify created compute instances and retrieve device UUIDs
nvidia-smi mig -lci
nvidia-smi -L
// 5. Query active MPS control daemon status (if evaluating software sharing)
echo "get_server_list" | nvidia-cuda-mps-control

Operational Troubleshooting for Shared GPU Environments

Rapid diagnosis of memory bus contention, daemon crashes, and allocation failures prevents cascading outages across shared inference nodes.

OneSource Cloud eliminates the performance penalties and architectural compromises of shared compute environments by delivering dedicated bare-metal GPU clusters. By providing full physical access to high-bandwidth NVLink fabrics, enterprise storage, and non-blocking networking, OneSource Cloud guarantees predictable latency and peak computational throughput.

FAQ

What is the primary architectural difference between NVIDIA MIG and CUDA MPS?

The primary architectural difference is hardware silicon partitioning versus software context multiplexing. NVIDIA MIG physically divides the GPU silicon into up to seven isolated instances with dedicated Streaming Multiprocessors, memory controllers, and L2 cache lines, ensuring strict fault isolation and guaranteed QoS. In contrast, CUDA MPS multiplexes multiple client processes into a shared CUDA context, maximizing SM core occupancy through concurrent kernel execution but leaving the global memory bus and VRAM pool vulnerable to contention.

When does an enterprise outgrow GPU slicing technologies and require OneSource Cloud dedicated bare metal?

An enterprise outgrows GPU slicing technologies when models exceed single-GPU memory capacity (requiring multi-GPU tensor parallelism over high-speed NVLink), when distributed training demands line-rate RoCE v2 collective communication, or when customer-facing inference SLOs cannot tolerate noisy-neighbor latency spikes. OneSource Cloud dedicated bare metal delivers unconstrained hardware access, eliminating slicing overhead and providing maximum throughput for mission-critical AI workloads.