AI data privacy patterns

idental leakage exits.

Step-by-Step Implementation

1. Define Privacy Policy Rules: Establish a declarative configuration mapping data types to actions (redact, hash, mask, or block). Rules should cover PII, PHI, financial identifiers, and internal secrets.
2. Implement Ingress Filtering: Intercept incoming requests, scan payloads for sensitive patterns, and apply transformations before prompt construction.
3. Enforce Context Minimization: Strip metadata, enforce token limits, and replace sensitive entities with stable pseudonyms to preserve semantic structure without exposing raw data.
4. Validate Egress Outputs: Scan model responses for accidental data reproduction, prompt leakage, or policy violations before returning to the client.
5. Log Privacy Events: Record sanitization actions, policy matches, and fallback triggers without storing raw payloads. Use cryptographic hashing for audit trails.

TypeScript Implementation

import { z } from 'zod';
import { createHash, randomUUID } from 'crypto';

// Privacy policy schema
const PrivacyRule = z.object({
  type: z.enum(['email', 'phone', 'ssn', 'credit_card', 'internal_id']),
  action: z.enum(['redact', 'hash', 'mask', 'block']),
  pattern: z.instanceof(RegExp),
  replacement: z.string().optional()
});

type PrivacyRule = z.infer<typeof PrivacyRule>;

// Stable pseudonym generator for context preservation
const pseudonymStore = new Map<string, string>();

function getPseudonym(value: string, type: string): string {
  const key = `${type}:${value}`;
  if (!pseudonymStore.has(key)) {
    pseudonymStore.set(key, `__${type.toUpperCase()}_${randomUUID().slice(0, 8)}__`);
  }
  return pseudonymStore.get(key)!;
}

// Core sanitization engine
export async function sanitizePayload(
  payload: string,
  rules: PrivacyRule[]
): Promise<{ sanitized: string; events: PrivacyEvent[] }> {
  const events: PrivacyEvent[] = [];
  let text = payload;

  for (const rule of rules) {
    const matches = text.matchAll(rule.pattern);
    for (const match of matches) {
      const raw = match[0];
      let replacement: string;

      switch (rule.action) {
        case 'redact':
          replacement = '[REDACTED]';
          break;
        case 'hash':
          replacement = createHash('sha256').update(raw).digest('hex').slice(0, 16);
          break;
        case 'mask':
          replacement = raw.slice(0, 3) + '****' + raw.slice(-3);
          break;
        case 'block':
          throw new PrivacyViolationError(`Blocked ${rule.type}: ${raw}`);
        default:
          replacement = getPseudonym(raw, rule.type);
      }

      text = text.replace(raw, replacement);
      events.push({ type: rule.type, action: rule.action, timestamp: Date.now() });
    }
  }

  return { sanitized: text, events };
}

// Egress validator
export function validateOutput(output: string, originalContext: string): ValidationResult {
  const leakage = detectContextReproduction(output, originalContext);
  return {
    safe: leakage.length === 0,
    violations: leakage,
    recommendation: leakage.length > 0 ? 'retry_with_stricter_prompt' : 'proceed'
  };
}

// Minimal type definitions for production use
interface PrivacyEvent {
  type: string;
  action: string;
  timestamp: number;
}

interface ValidationResult {
  safe: boolean;
  violations: string[];
  recommendation: string;
}

class PrivacyViolationError extends Error {
  constructor(message: string) {
    super(message);
    this.name = 'PrivacyViolationError';
  }
}

Architecture Decisions and Rationale

Gateway Placement: The sanitization layer sits between your orchestration framework (LangChain, LlamaIndex, or custom) and the LLM provider. This ensures policy enforcement is provider-agnostic and survives model swaps.

Pseudonymization over Redaction: Complete redaction breaks semantic relationships in prompts. Replacing entities with stable pseudonyms preserves context structure while eliminating exposure. The in-memory map can be swapped for Redis with TTL expiration in distributed deployments.

Zero Raw Data Logging: Audit events store only policy matches, actions taken, and cryptographic fingerprints. This satisfies compliance requirements without creating secondary data stores that become breach targets.

Egress Validation: Models occasionally regurgitate context when prompted adversarially or when temperature settings are misconfigured. Output validation catches reproduction patterns before they reach end users.

Policy-as-Code: Rules are version-controlled, tested, and deployed alongside application code. This eliminates drift between legal requirements and runtime behavior.

Pitfall Guide

1. Regex-Only PII Detection: Regular expressions fail on contextual, semantic, or obfuscated data. LLMs can infer sensitive information from partial patterns, abbreviations, or structural cues. Production systems require NLP-based detectors or rule engines with confidence scoring.

2. Assuming Anonymization Equals Compliance: Stripping names or IDs is insufficient. Re-identification attacks combine quasi-identifiers (timestamps, locations, behavioral patterns) to reconstruct identities. True compliance requires differential privacy guarantees or strict data minimization.

3. Ignoring Prompt Injection as a Privacy Vector: Attackers can craft prompts that force models to output training data, system instructions, or previously processed context. Privacy patterns must include input validation, instruction separation, and output filtering.

4. Logging Full Prompts for Debugging: Development teams frequently enable verbose logging to troubleshoot hallucinations. This creates unencrypted data lakes that violate retention policies. Use sampled logging, hash-based correlation, or synthetic test data for debugging.

5. Trusting Cloud Provider Privacy Claims Without SLAs: Default cloud AI terms often include data retention for model improvement. Even with opt-out toggles, auditability is limited. Contractual data processing agreements and explicit data residency clauses are mandatory for regulated workloads.

6. Treating Synthetic Data as Identical to Real Data: Synthetic datasets preserve statistical distributions but lose edge-case behavior. Models trained or fine-tuned on synthetic data may exhibit degraded performance on rare but critical scenarios. Validate distributional parity before production rollout.

7. Missing Key Rotation for Vector Stores: Embedding databases store semantic representations of sensitive documents. If encryption keys aren't rotated or if vector search returns raw metadata alongside embeddings, privacy controls are bypassed. Envelope encryption and metadata stripping are non-negotiable.

Best Practices from Production:

• Implement defense-in-depth: combine ingress filtering, context minimization, egress validation, and audit logging.
• Use policy-as-code with automated compliance testing in CI/CD.
• Enforce zero-trust data flow: assume every component can be compromised and limit data exposure accordingly.
• Conduct regular red-team exercises focused on prompt extraction and context leakage.
• Maintain separate execution environments for PII processing and model inference.

Production Bundle

Action Checklist

• Define privacy policy rules: Map all data types to redaction, hashing, masking, or blocking actions using declarative configuration.
• Implement ingress sanitization: Deploy middleware to scan and transform payloads before prompt construction.
• Enforce context minimization: Strip metadata, enforce token limits, and replace sensitive entities with stable pseudonyms.
• Validate egress outputs: Scan model responses for context reproduction, prompt leakage, or policy violations.
• Configure audit telemetry: Log privacy events with cryptographic hashing; never store raw prompts or responses.
• Assess vendor contracts: Verify data processing agreements, retention policies, and residency requirements with AI providers.
• Schedule red-team validation: Test prompt injection, context extraction, and adversarial leakage quarterly.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Customer Support Chat	Privacy-Enhanced Gateway	Balances latency, compliance, and model capability; handles high volume with consistent policy enforcement	Low (+15-20% infra cost)
Internal Knowledge Base	Federated Learning + Gateway	Keeps raw documents on-premise; enables model training without centralizing sensitive data	Medium (+35% orchestration cost)
Healthcare AI (PHI)	On-Device Inference	Eliminates network exposure; meets HIPAA strictest requirements; accepts higher latency	High (+80% hardware cost)
Batch Analytics Pipeline	Synthetic Data Generation	Removes PII entirely from training/analysis; preserves statistical validity for insights	Low-Medium (+10% data processing cost)

Configuration Template

privacy_gateway:
  version: "1.0"
  ingress:
    enabled: true
    rules:
      - type: "email"
        pattern: "\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Z|a-z]{2,}\\b"
        action: "hash"
      - type: "phone"
        pattern: "\\b\\+?\\d{1,3}[-.\\s]?\\(?\\d{1,4}\\)?[-.\\s]?\\d{1,4}[-.\\s]?\\d{1,9}\\b"
        action: "mask"
      - type: "internal_id"
        pattern: "INT-[A-Z0-9]{6,10}"
        action: "pseudonymize"
    max_context_tokens: 4000
    strip_metadata: true
  egress:
    enabled: true
    leakage_threshold: 0.85
    retry_on_violation: true
    max_retries: 2
  audit:
    enabled: true
    store: "redis"
    ttl_hours: 72
    hash_algorithm: "sha256"
    log_raw_payload: false
  fallback:
    model: "local-small"
    trigger: "policy_violation"
    timeout_ms: 1500

Quick Start Guide

1. Install dependencies: npm install zod crypto @types/node
2. Create policy file: Save the YAML template as privacy-policy.yaml and adjust patterns/actions to match your data types.
3. Wrap your LLM client: Replace direct model calls with the sanitizePayload() and validateOutput() functions from the core solution. Route through the gateway middleware before and after inference.
4. Run validation: Execute a test suite with synthetic PII samples. Verify that ingress filtering transforms data, egress validation blocks leakage, and audit logs record events without storing raw content. Deploy to staging and monitor latency overhead against your SLA thresholds.