Power Delivery — infra.bea.ai

AI GPU power has grown 3× in three generations (A100: 400W → H100: 700W → B200: 1000W), pushing rack densities from 10kW to 100kW+. Traditional data centers designed for 5-10kW racks simply cannot host modern GPU infrastructure without major upgrades.

1. GPU Power Evolution

GPU	Year	TDP (W)	Rack of 8-GPU servers	Notes
V100	2017	300	~15 kW (4 servers)	Air-cooled
A100 SXM	2020	400	~20 kW (4 servers)	Air-cooled (tight)
H100 SXM	2022	700	~40 kW (4 servers)	DLC recommended
H200 SXM	2024	700	~40 kW (4 servers)	DLC recommended
B200 SXM	2024	1000	~55 kW (4 servers)	DLC required
GB200 NVL72	2025	1200/GPU	~120 kW (72 GPUs/rack)	DLC required, liquid manifold

Server-Level Power

DGX H100 (8× H100 SXM):
  GPUs:      8 × 700W = 5,600W
  CPUs:      2 × 350W =   700W
  Memory:              =   200W
  NVMe:                =   100W
  NICs (8):            =   200W
  Fans/misc:           =   200W
  ────────────────────────────
  Total:              ~7,000W (7.0 kW)

  4 servers per rack = ~28 kW (GPU only)
  + networking switches = ~32 kW per rack

2. Power Delivery Chain

Utility grid (medium voltage: 13.8-69 kV)
  │
  ├─→ Transformer (step down to 480V AC)
  │
  ├─→ UPS (Uninterruptible Power Supply)
  │     ├── Battery backup: 5-15 minutes (enough for generator start)
  │     └── Diesel generator kicks in for sustained outage
  │
  ├─→ PDU (Power Distribution Unit) — per-row or per-rack
  │     ├── Step down to 208V or 400V AC
  │     └── Monitor power per circuit
  │
  └─→ Server PSU (Power Supply Unit) — per-server
        ├── AC → DC conversion (12V)
        └── Feeds GPU, CPU, memory, NVMe

Redundancy Configurations

Configuration	Description	Availability	Cost
N	Single power path	99.9%	Baseline
N+1	One spare component per path	99.99%	+20%
2N	Fully redundant paths (A+B feed)	99.999%	+80-100%
2N+1	Redundant paths + spare	99.9999%	+100%+

Most AI clusters use N+1 or 2N depending on workload criticality. Training can tolerate brief outages (checkpoint + restart). Serving requires 2N for SLA compliance.

3. Facility Constraints

Most existing colocation facilities were built for 5-10 kW/rack:

Constraint	Traditional DC	AI-ready DC
Rack power	5-10 kW	30-120 kW
Cooling per rack	Air (5-10 kW)	DLC (30-120 kW)
Floor loading	100 kg/m²	300+ kg/m² (heavier servers)
Power per MW	Abundant	Scarce (utility limits)
Cable density	Moderate	Extreme (NVLink + IB per rack)

Upgrading Existing Facilities

Can the facility support AI GPUs?
├── Power capacity sufficient? (MW available)
│   ├── Yes → Check cooling
│   └── No → Utility upgrade needed (6-18 months)
├── Cooling supports 30+ kW/rack?
│   ├── Yes → Check floor loading
│   └── No → DLC retrofit needed (3-6 months)
├── Floor can handle weight?
│   ├── Yes → Proceed
│   └── No → Structural reinforcement (rare, expensive)
└── Cable pathway capacity?
    ├── Yes → Proceed
    └── No → Additional cable trays (weeks)

4. Power Monitoring

# Per-GPU power monitoring via DCGM
gpu_power = {
    "DCGM_FI_DEV_POWER_USAGE": "current draw (W)",
    "DCGM_FI_DEV_POWER_LIMIT": "TDP limit (W)",
    "DCGM_FI_DEV_ENFORCED_POWER_LIMIT": "throttle point (W)",
}

# Rack-level monitoring via smart PDU
rack_power = {
    "total_kw": 38.5,
    "per_circuit_kw": [9.8, 9.6, 9.7, 9.4],  # 4 servers
    "peak_kw_last_hour": 41.2,
    "power_factor": 0.98,
}

# Alert if approaching breaker limit
if rack_power["total_kw"] > rack_breaker_limit * 0.9:
    alert("Rack power at 90% of breaker capacity")

5. Power Optimization

Technique	Savings	Implementation
GPU power capping	10-20% power, 5-10% perf loss	`nvidia-smi -pl 600` (cap H100 to 600W)
Dynamic frequency scaling	5-15% power	NVIDIA power management modes
Idle GPU power-down	30-50% idle power	Enable GPU idle states
High-efficiency PSUs (Titanium)	2-5% total	96%+ efficiency PSUs
Higher voltage distribution (400V)	1-3% total	Fewer conversion stages

Self-Check

Calculate rack power for 4× DGX H100 servers including overhead
Design a power delivery chain with N+1 redundancy
Assess whether an existing 10kW/rack facility can host H100s
Set up per-GPU and per-rack power monitoring with alerts
Estimate power savings from capping H100 TDP from 700W to 600W