Cooling Systems — infra.bea.ai

Cooling is the physical constraint that determines how many GPUs you can pack into a rack. Air cooling maxes out at ~25 kW/rack. Modern GPU racks need 40-120 kW. Direct liquid cooling (DLC) is no longer optional for dense AI deployments — it’s a requirement.

1. Cooling Capacity by Method

Method	Max rack power	PUE	Upfront cost	Retrofit difficulty
Air cooling (CRAC/CRAH)	15-25 kW	1.3-1.6	Low	N/A (default)
Rear-door heat exchanger	25-40 kW	1.1-1.3	Medium	Easy (bolt-on)
Direct liquid cooling (DLC)	60-120+ kW	1.05-1.15	High	Moderate (piping)
Immersion cooling	100+ kW	1.02-1.08	Highest	Hard (tanks)

2. Air Cooling

Hot air from servers rises, gets cooled by CRAC/CRAH units, and recirculates:

Server → Hot aisle (35-45°C) → CRAH unit → Cold aisle (18-27°C) → Server

Optimization

Technique	Benefit
Hot/cold aisle containment	Prevents mixing, +2-3 kW/rack capacity
Raised floor with directed airflow	Better distribution
Variable-speed fans	Reduce energy at low load
Higher inlet temperature (27°C vs 18°C)	Reduce cooling energy, more free cooling hours

Hard limit: Air cooling fundamentally cannot handle >~25 kW/rack. The air can’t carry enough heat away. For H100/B200 racks, air alone is insufficient.

3. Direct Liquid Cooling (DLC)

Cold plates on GPUs and CPUs carry heat to a Coolant Distribution Unit (CDU):

GPU cold plate → warm coolant (40-45°C) → CDU → cooling tower/chiller → cold coolant (25-30°C) → GPU

Why DLC for AI

Advantage	Impact
3-5× cooling capacity per rack	Supports 40-120 kW racks
GPU runs cooler (50-65°C vs 75-85°C)	Less thermal throttling, longer lifespan
Lower PUE (1.05-1.15)	15-20% less total power
Quieter (fewer fans)	Better working environment
Smaller footprint	More racks per floor

DLC Components

Per-rack:
  ├── Cold plates (on each GPU + CPU): $500-1000/server
  ├── Quick-connect manifolds: $2,000-5,000/rack
  └── Flexible hoses: $500-1,000/rack

Per-row:
  └── CDU (Coolant Distribution Unit): $30,000-80,000
      ├── Heat exchanger (liquid-to-liquid or liquid-to-air)
      ├── Pumps (redundant)
      └── Monitoring (flow rate, temperature, pressure)

Facility:
  └── Cooling tower or chiller plant connection
      ├── Piping infrastructure
      └── Water treatment system

GB200 NVL72 Rack Cooling

GB200 NVL72 rack: 120 kW total
  ├── 72 GPUs with integrated cold plates
  ├── 36 Grace CPUs with cold plates  
  ├── NVSwitch trays with cold plates
  └── Single-phase liquid cooling manifold at rack level

Coolant: facility water (25-45°C) or propylene glycol mix
Flow rate: ~60-80 liters/minute per rack

4. Rear-Door Heat Exchangers (RDHx)

A retrofit option that extends air-cooled racks to ~40 kW:

Server exhausts hot air → rear-door water coil captures heat → cooled air exits

Pros: Easy retrofit (replace rear door), no server modification.
Cons: Limited to ~40 kW, still needs some room air cooling, water near servers.

5. Immersion Cooling

Servers submerged in dielectric fluid:

Single-phase: Servers in fluid bath → fluid circulated to external heat exchanger
Two-phase: Fluid boils on hot components → vapor condenses on coils → drips back

Property	Single-phase	Two-phase
Fluid	Mineral oil, synthetic	Engineered fluids (3M Novec)
Max density	100+ kW/tank	100+ kW/tank
PUE	1.03-1.06	1.02-1.05
Maintenance	Messy (oil)	Cleaner but expensive fluid
Server compatibility	Standard servers (with mods)	Requires specific design
Cost per kW	High	Very high

Best for: Extreme density, harsh environments (dust, heat), or locations without water.

6. Decision Matrix

Rack power needed:
├── <15 kW (T4, light CPU):      Air cooling sufficient
├── 15-25 kW (A100, sparse):     Air + hot/cold containment
├── 25-40 kW (H100, moderate):   Rear-door heat exchanger OR DLC
├── 40-80 kW (H100 dense):       DLC required
└── 80-120+ kW (GB200 NVL72):    DLC required (factory-integrated)

New build vs retrofit:
├── New construction → DLC from day 1 (easiest, cheapest long-term)
├── Existing facility → RDHx for quick upgrade, DLC for long-term
└── Extreme density → Immersion if floor loading supports it

Self-Check

Calculate why air cooling fails above 25 kW/rack
Design a DLC system for a row of H100 racks including CDU and piping
Compare RDHx vs DLC for retrofitting an existing air-cooled facility
Estimate PUE improvement from switching air cooling to DLC
Evaluate immersion cooling for a GB200 deployment

References

NVIDIA DLC Deployment Guide
ASHRAE TC9.9: Thermal Guidelines for Data Processing Environments
For power delivery, see Power Delivery
For rack layout, see Physical Topology