Confidential Computing — infra.bea.ai

Confidential computing protects data during processing — not just at rest (encryption) or in transit (TLS). For AI, this means model weights and user prompts are encrypted even from the infrastructure operator. A cloud provider running your model can’t read the weights; a model provider processing your queries can’t see the prompts.

1. The Trust Problem

Traditional AI deployment trusts the infrastructure operator:

Traditional:
  User data (plaintext) → Cloud GPU (operator can inspect) → Response

Confidential:
  User data (encrypted) → TEE enclave (operator CANNOT inspect) → Response (encrypted)

Scenario	Threat	Confidential computing solves
Enterprise using cloud LLM	Cloud provider reads confidential prompts	Prompts encrypted inside TEE
API provider hosting model	Customers extract model weights	Weights encrypted in TEE
Multi-tenant GPU sharing	Tenant A reads tenant B’s data	Hardware-isolated memory per tenant
Regulated industry	Auditor needs proof data was protected	Remote attestation provides proof

2. TEE Technologies

CPU-Based TEEs

Technology	Vendor	Protection	Overhead	Status
Intel TDX	Intel	Full VM encryption (memory, registers)	5-15%	GA on 4th Gen Xeon+
AMD SEV-SNP	AMD	Full VM encryption + integrity	5-10%	GA on EPYC Milan+
Intel SGX	Intel	Per-process enclaves	10-30%	Older, limited enclave size
ARM CCA	ARM	Realm-based isolation	5-15%	Emerging

GPU Confidential Computing

NVIDIA Confidential Computing (H100+):
  ├── GPU memory is encrypted (AES-256)
  ├── CPU↔GPU transfers over encrypted PCIe channel
  ├── GPU firmware attested (verified before use)
  └── Operator cannot read GPU memory via MMIO/debugger

NVIDIA H100 CC Mode:

# Enable Confidential Computing mode
nvidia-smi conf-compute -igs 1  # Enable input guard on GPU 1
nvidia-smi conf-compute -eas 1  # Enable encryption attestation

# Verify CC mode is active
nvidia-smi conf-compute -q
# Output: CC Status: ON, Key Rotation: Enabled

3. Attestation: Proving the Enclave Is Real

Attestation lets a remote party verify that:

The hardware is genuine (not emulated)
The software inside the enclave is the expected version
The enclave hasn’t been tampered with

# Attestation flow
async def verify_enclave(enclave_url: str, expected_measurements: dict):
    """Verify that the remote enclave is genuine and running expected code."""
    
    # 1. Request attestation report from enclave
    report = await httpx.get(f"{enclave_url}/attestation-report")
    
    # 2. Verify hardware signature (from Intel/AMD/NVIDIA root of trust)
    if not verify_hardware_signature(report.hardware_quote):
        raise AttestationError("Hardware signature verification failed")
    
    # 3. Check software measurements match expected values
    if report.firmware_hash != expected_measurements["firmware"]:
        raise AttestationError("Firmware measurement mismatch")
    
    if report.application_hash != expected_measurements["application"]:
        raise AttestationError("Application measurement mismatch")
    
    # 4. Establish encrypted channel using enclave's attested public key
    session_key = establish_tls(report.public_key)
    return session_key

4. Deploying AI in Confidential VMs

Azure Confidential VMs with GPU

# Azure: Deploy H100 confidential VM
az vm create \
    --name llm-confidential \
    --size Standard_NC40ads_H100_v5 \
    --image Canonical:0001-com-ubuntu-confidential-vm-jammy:22_04-lts-cvm:latest \
    --security-type ConfidentialVM \
    --os-disk-security-encryption-type VMGuestStateOnly \
    --enable-vtpm true \
    --enable-secure-boot true

Performance Overhead

Workload	Without CC	With CC	Overhead
LLM inference (70B, H100)	2,500 tok/s	2,250 tok/s	~10%
Embedding (batch)	8,000 chunks/s	7,200 chunks/s	~10%
Training (all-reduce heavy)	Baseline	-5 to -15%	Higher for comm-heavy

The overhead comes primarily from memory encryption and CPU-GPU secure channel establishment, not from the computation itself.

5. Limitations and Tradeoffs

Limitation	Impact	Mitigation
Performance overhead	5-15% throughput loss	Accept for security-critical workloads
GPU support	H100+ only, no multi-GPU CC across NVLink	Use single-GPU or TP within CC boundary
Debugging	Can’t inspect enclave memory (by design)	Log within enclave, export encrypted logs
Ecosystem	Limited framework support	Growing: PyTorch + NVIDIA CC SDK
Side-channels	Timing attacks, power analysis still possible	Research ongoing, not fully mitigated
Key management	Enclave needs access to model/data keys	Use remote KMS with attestation-gated access

6. When to Use Confidential Computing

Is this a regulated industry (healthcare, finance, government)?
├── Yes → Strongly consider CC for compliance
└── No
    ├── Are prompts highly confidential?
    │   ├── Yes → CC protects from infrastructure operator
    │   └── No → Standard encryption sufficient
    ├── Are model weights a trade secret?
    │   ├── Yes → CC prevents weight extraction
    │   └── No → Standard deployment fine
    └── Is this multi-tenant with untrusted co-tenants?
        ├── Yes → CC + MIG for strongest isolation
        └── No → MIG or dedicated GPUs sufficient

Self-Check

Explain the trust model difference between traditional and confidential AI deployment
Compare Intel TDX, AMD SEV-SNP, and NVIDIA CC on protection and overhead
Describe the attestation flow and what it proves
Estimate the performance overhead of confidential computing for LLM inference
Decide when confidential computing is worth the overhead