Networking Capacity Planning for GPU Cluster Scale

Networking capacity planning for GPU clusters sizes the fabric to handle collective operations at the cluster's full GPU count — bandwidth must scale with GPUs, topology must avoid oversubscription that creates congestion, and the fabric must be planned for growth so it does not become the bottleneck when more GPUs are added. For the networking design requirements, see networking requirements for AI. For the cluster evaluation, see evaluating AI cluster networking.

Capacity Planning Method

Step 1: Model collective operation bandwidth — all-reduce runs on every training step, and its bandwidth requirement scales with GPU count and model size. Size the fabric to sustain this bandwidth at peak concurrency, not at idle. Step 2: Plan topology for the target scale — a flat, non-oversubscribed topology where GPU pairs communicate at full link speed. As GPU count grows, the switch hierarchy must scale without introducing oversubscription. Step 3: Set oversubscription limits — oversubscription saves cost but caps performance. The limit is workload-dependent: training tolerates less oversubscription than inference because collective operations demand more bandwidth. Step 4: Plan for growth — the fabric must accommodate the GPU count you will have, not just the count you start with. An undersized fabric throttles the GPUs added later. For the cost estimation, see GPU networking cost estimation.

FAQ

How do I plan network capacity for a GPU cluster?

Size bandwidth for collective operations at the full GPU count, plan a non-oversubscribed topology, set workload-appropriate oversubscription limits, and plan for growth. The fabric must scale with GPUs. See the four steps above.

Summary

GPU network capacity planning sizes for collective ops at scale, with topology and growth planning. For the full framework, see networking requirements.