Compliance

AI infrastructure faces a growing web of regulations — EU AI Act, GDPR, HIPAA, SOC 2 — each requiring specific controls around data handling, model transparency, and audit trails. Compliance isn’t a one-time checkbox; it’s an infrastructure capability built into logging, storage, and deployment systems.

---

1. Audit Logging

Every model interaction must be logged immutably for post-incident investigation and regulatory review:

class ComplianceAuditLog:
    """Append-only, tamper-evident audit log."""
    
    async def log_inference(self, event: InferenceEvent):
        entry = {
            "event_id": uuid4().hex,
            "timestamp": datetime.utcnow().isoformat(),
            "event_type": "inference",
            
            # Who
            "tenant_id": event.tenant_id,
            "user_id": event.user_id,
            "api_key_id": event.key_id,
            "ip_address": event.ip_address,
            
            # What
            "model": event.model,
            "model_version": event.model_version,
            "input_tokens": event.input_tokens,
            "output_tokens": event.output_tokens,
            "prompt_hash": hashlib.sha256(event.prompt.encode()).hexdigest(),
            
            # Where
            "region": event.region,
            "gpu_id": event.gpu_id,
            
            # Safety
            "guardrail_triggered": event.guardrail_triggered,
            "content_filtered": event.content_filtered,
        }
        
        # Append-only storage (not modifiable by application)
        await self.immutable_store.append(entry)
        
        # Optional: hash chain for tamper detection
        entry["prev_hash"] = self.last_hash
        entry["entry_hash"] = hashlib.sha256(
            json.dumps(entry, sort_keys=True).encode()
        ).hexdigest()
        self.last_hash = entry["entry_hash"]

Retention Policies

Data type	Retention	Regulation
Audit logs (metadata)	7 years	SOC 2, financial regulations
Prompt/completion (if stored)	30 days or as required	GDPR right to deletion
Model training data provenance	Indefinite	EU AI Act
Billing records	7 years	Tax/financial regulations
Security event logs	1 year	SOC 2, ISO 27001

---

2. Data Residency

Keep data in specific geographic regions to comply with local regulations:

class DataResidencyRouter:
    REGION_MAPPING = {
        "EU": {
            "inference_endpoints": ["eu-west-1.llm.internal"],
            "storage_bucket": "s3://llm-data-eu-west-1",
            "logging_endpoint": "eu-west-1.logs.internal",
        },
        "US": {
            "inference_endpoints": ["us-east-1.llm.internal"],
            "storage_bucket": "s3://llm-data-us-east-1",
            "logging_endpoint": "us-east-1.logs.internal",
        },
    }
    
    def route_request(self, tenant: Tenant) -> RegionConfig:
        region = tenant.data_residency_region  # "EU", "US", etc.
        config = self.REGION_MAPPING[region]
        
        # Verify: prompt data NEVER leaves the specified region
        # This means: inference GPU must be in-region, logs stored in-region
        return config

What Must Stay In-Region

Data	Must be in-region?	Notes
User prompts	Yes (GDPR, data sovereignty)	Cannot cross borders
Model responses	Yes	Contains processed personal data
Audit logs	Yes	Evidence must be locally stored
Model weights	Usually no	Unless custom-trained on local data
Aggregated metrics	Usually no	If properly anonymized

---

3. Model Lineage Tracking

The EU AI Act requires knowing what data trained a model, what evaluations it passed, and its full deployment history:

class ModelLineage:
    """Track the full lifecycle of a model for regulatory compliance."""
    
    async def register_model(self, model: ModelSpec):
        lineage = {
            "model_id": model.id,
            "model_version": model.version,
            "created_at": datetime.utcnow().isoformat(),
            
            # Training provenance
            "base_model": model.base_model,          # e.g., "meta-llama/Llama-3.1-70B"
            "training_data": {
                "datasets": model.training_datasets,  # List of dataset IDs
                "data_licenses": model.data_licenses,
                "pii_handling": model.pii_handling,    # "removed", "anonymized", "present"
                "cutoff_date": model.data_cutoff,
            },
            
            # Fine-tuning
            "fine_tune_data": model.fine_tune_datasets,
            "fine_tune_method": model.fine_tune_method,  # "SFT", "RLHF", "DPO"
            
            # Evaluation
            "evaluations": [
                {"benchmark": "MMLU", "score": 0.82, "date": "2024-06-01"},
                {"benchmark": "safety_eval", "score": 0.99, "date": "2024-06-01"},
                {"benchmark": "bias_audit", "score": "pass", "date": "2024-06-01"},
            ],
            
            # Deployment
            "deployment_regions": ["us-east-1", "eu-west-1"],
            "risk_classification": "limited",  # EU AI Act: minimal/limited/high/unacceptable
            "approved_by": "ml-safety-team@company.com",
        }
        
        await self.registry.store(lineage)

---

4. EU AI Act Compliance

The EU AI Act classifies AI systems by risk level:

Risk level	Examples	Requirements
Unacceptable	Social scoring, real-time facial recognition	Banned
High	Medical diagnosis, hiring, credit scoring	Full compliance package
Limited	Chatbots, content generation	Transparency obligations
Minimal	Spam filters, game AI	No specific requirements

High-Risk Requirements

If your AI system is classified as high-risk:

# Required documentation and controls
high_risk_compliance:
  risk_management:
    - Risk assessment document
    - Mitigation measures for identified risks
    - Continuous monitoring plan
  
  data_governance:
    - Training data documentation
    - Bias testing on training data
    - Data quality measures
  
  technical_documentation:
    - Model architecture description
    - Training methodology
    - Performance metrics and limitations
    - Intended use and misuse scenarios
  
  transparency:
    - Clear disclosure: "This response was generated by AI"
    - Users informed of AI system capabilities and limitations
    - Contact information for human oversight
  
  human_oversight:
    - Human review process for high-stakes decisions
    - Ability to override AI decisions
    - Escalation procedures
  
  accuracy_and_robustness:
    - Regular evaluation on representative benchmarks
    - Adversarial testing (jailbreak, bias probes)
    - Performance monitoring in production

---

5. Right to Deletion (GDPR Article 17)

When a user requests deletion, remove their data from all systems:

async def handle_deletion_request(user_id: str):
    """GDPR right to be forgotten — delete all user data."""
    
    # 1. Delete from inference logs (where prompts/completions are stored)
    await audit_log.delete_user_data(user_id)
    
    # 2. Delete from vector databases (if user content was embedded)
    await vector_db.delete(filter={"user_id": user_id})
    
    # 3. Delete from caches
    await cache.delete_pattern(f"*:{user_id}:*")
    
    # 4. Delete from fine-tuning datasets (if user data was used)
    await dataset_manager.remove_user_data(user_id)
    
    # 5. Log the deletion (this log entry is retained for compliance)
    await audit_log.log_event({
        "event_type": "data_deletion",
        "user_id": user_id,
        "timestamp": datetime.utcnow().isoformat(),
        "systems_cleaned": ["audit_log", "vector_db", "cache", "datasets"],
    })
    
    # 6. If user data was in training data: flag for next model retrain
    await model_registry.flag_contaminated(user_id)

---

Self-Check

• Design an append-only audit log with tamper detection
• Implement data residency routing that keeps prompts in-region
• Build a model lineage record covering training data, evaluations, and deployment
• Classify an AI system under the EU AI Act risk framework
• Implement GDPR right-to-deletion across all data stores

---

References

• EU AI Act Text
• GDPR Official Text
• SOC 2 Compliance Guide
• For audit logging in the gateway, see API Gateway