Liquid Cooling for AI Data Centers: Power Density Limits

Liquid cooling is a heat-removal design that uses fluid, not only room air, to keep high-TDP GPUs inside their thermal envelope. It becomes a facilities decision when rack power density outruns what raised-floor air and conventional CRAH units can reject without throttling the accelerators.

Hopper-class H200 boards are commonly specified up to about 700W TDP. Blackwell-class B200 boards are commonly specified up to about 1000W TDP. Those are device envelopes, not rack totals. A dense training node multiplies them, then adds CPUs, NICs, and power-supply loss. Air can still work at lower density. It does not automatically work at the densities buyers now request.

This article explains where air hits the wall, how the three common liquid approaches differ, and what operations change after the pipes go in.

Why Air Stops Being Enough

Air cooling fails in two ways that look like GPU problems. Inlet temperature rises, fans scream, and the device clocks down. Or the room can still cool a few racks but cannot cool a row of them without hot-aisle collapse. In both cases training step time grows and inference tail latency moves. The ticket says "GPU performance." The cause is heat rejection.

Power density is the planning number, not a marketing PUE. Ask for kW per rack and for the number of racks in the row, then ask whether the hall was designed for that heat. A colo that once hosted 8–12 kW enterprise racks is not automatically an AI hall because someone hung a rear-door exchanger on one cabinet.

Do not treat vendor PUE slides as a promise for your row. PUE depends on climate, load factor, and how the plant is run. Use it as a conversation, then ask for measured values at the density you will actually occupy.

Three Liquid Approaches, Three Operating Models

Rear-door units help when the room is close to its air limit and you need a path that existing operations staff can learn. They do not remove the fact that the GPU still depends on internal airflow. Direct-to-chip is the common choice for the current high-TDP training nodes because it takes heat at the package. Immersion can support very high density, but it changes how you rack, cable, and replace a failed board. Pick the model you can operate, not the model that looks densest on a slide.

Facility Water and What Breaks First

Liquid loops fail at the plant more often than at the cold plate. CDUs need capacity, redundancy, and water chemistry that matches the vendor spec. A leak-detection plan is part of the design, not an afterthought. So is a procedure for isolating a rack without taking the row down.

Ask where heat goes after the CDU. If the building condenser plant is already at its limit, adding GPU liquid loops just moves the bottleneck outdoors. Power and cooling are one constraint. A hall that can deliver 40 kW to a rack but cannot reject 40 kW is not ready.

Networking and storage still sit in the same room. High-performance AI networking designs that pack switches beside GPU trays change airflow and service clearances. Cooling plans that ignore the fabric will be revised after the first install.

What This Means for GPU Selection and Placement

H200-class racks can be air-cooled in some halls and liquid-cooled in others. The deciding inputs are inlet spec, rack kW, and how tightly you pack the row. B200-class nodes at the high end of the TDP range push more designs toward D2C or a purpose-built hall. Neither statement is a universal rule. It is a site survey plus a node BOM.

If you are comparing generations on memory and power, read that against cooling, not only against list TDP. A faster GPU that throttles is a slower GPU. The companion comparison of memory, power, and training cost belongs with the node; the cooling decision belongs with the hall.

Colocation that cannot provide facility water, CDU space, or the contracted kW will force you into a purpose-built room or a provider that already built one. That is a placement decision, not a preference for pipes.

Operations After the Loop Is Live

Liquid cooling adds work that GPU teams do not do in an air-only lab: fluid inventory, hose inspection, CDU alarms, and vendor-approved board replacement. Managed AI infrastructure is often the difference between a successful first rack and a row that operations refuses to touch.

Monitor GPU clocks, memory temperature, and inlet or coolant temperature together. Thermal throttling that you only see in step time is already late. Treat cooling alarms with the same severity as a failed NVLink or a dropped storage mount.

U.S. sites still have to meet the same power and water reality. A Texas hall can be a fit when the plant was designed for AI density. It is not a fit because of the state name. Verify contracted kW, water, and the last time a full rack was run at the stated load. OneSource Cloud places clusters only where those checks can be answered with a hall design, not a brochure PUE.

Serving Decision Matrix: Enterprise LLM Inference Infrastructure

Deploying latency-sensitive large language models at enterprise scale requires infrastructure engineered for steady-state throughput and microsecond-level tail latency guarantees. On OneSource Dedicated Private GPU Cloud infrastructure, inference pipelines execute on dedicated bare-metal instances where GPU memory, PCIe bandwidth, and tensor cores are 100% isolated from third-party contention. By eliminating the hypervisor scheduling jitter that plagues multi-tenant cloud environments, OneSource enables production serving frameworks (such as vLLM and TensorRT-LLM) to sustain high token generation rates and tight P99 latency SLAs even during peak concurrent request bursts.

FAQ

Do H200 and B200 GPUs require liquid cooling?

Not in every installation. They require a hall that can reject the rack heat at the density you chose. Many H200 deployments still use air or rear-door assist. High-density B200 nodes more often need direct-to-chip or a purpose-built plant. Confirm against the node BOM and the site, not a slogan.

What is the difference between rear-door and direct-to-chip cooling?

Rear-door exchangers cool the exhaust air leaving the rack. Direct-to-chip puts cold plates on the devices and moves most of the heat in liquid. D2C usually supports higher device TDP. Rear-door is closer to an air-hall retrofit.

Does liquid cooling guarantee a better PUE?

No. It can improve heat rejection at high density, which may improve PUE if the plant is designed and loaded well. Climate, idle racks, and poor controls can erase the gain. Ask for measured PUE at your density.

Can we retrofit liquid cooling into existing colocation?

Sometimes, if the colo offers facility water, floor loading, CDU space, and a leak process. Many legacy halls cannot. Retrofit cost then exceeds a move to a purpose-built AI room.

What should we monitor after installing liquid cooling?

Coolant temperature and flow, CDU health, leak sensors, GPU memory and hot-spot temperatures, and clocks. Correlate those with job-step time so throttling is visible as an infrastructure event, not only as a model complaint.

Why deploy latency-sensitive LLM inference on OneSource private GPUs?

OneSource private GPU infrastructure delivers 100% dedicated bare-metal compute and VRAM, completely isolated from cross-tenant contention. This eliminates hypervisor scheduling jitter and shared-network packet collisions, ensuring deterministic P99 tail latency, sustained token throughput, and optimal tensor parallel scaling for production enterprise LLM serving.

Summary

Liquid cooling is how AI halls keep high-TDP GPUs from becoming heat-limited. Choose rear-door, direct-to-chip, or immersion based on density, facility water, and the operations team you actually have. Treat power and heat rejection as one constraint, and verify kW and water on the specific row.

OneSource Cloud places enterprise GPU clusters in U.S. facilities designed for AI density rather than forcing a legacy air hall to pretend. Request an architecture review to match H200 or B200 node power to a hall that can reject it.