AI-powered translation

sacrificing quality.

Core Solution

Architecture Overview

A production AI translation service requires four layers:

1. Context Extractor: Gathers surrounding UI text, tone guidelines, and variable definitions.
2. Router & Guardrails: Determines if a request needs AI translation or can use a fallback/static map. PII redaction occurs here.
3. Semantic Cache: Stores translations keyed by vector embeddings of the content + context.
4. LLM Provider: Executes the translation with optimized prompts.

Step-by-Step Implementation

1. Context Schema Definition

Define a structured context object to ensure consistency.

export interface TranslationContext {
  sourceLocale: string;
  targetLocale: string;
  tone: 'formal' | 'casual' | 'technical';
  domain: 'finance' | 'healthcare' | 'general';
  surroundingText?: string[]; // Contextual hints
  variables?: Record<string, string>;
}

2. Semantic Cache Key Generation

Use a hashing strategy that incorporates embeddings. For implementation without a vector DB, a simplified approach uses a normalized hash of the text and context, or integrates with a service like Redis with vector search.

import { createHash } from 'crypto';

function generateCacheKey(
  text: string, 
  context: TranslationContext
): string {
  // Normalize text: trim, lowercase, remove extra whitespace
  const normalizedText = text.trim().toLowerCase().replace(/\s+/g, ' ');
  
  // Serialize context for deterministic hashing
  const contextString = JSON.stringify({
    tone: context.tone,
    domain: context.domain,
    locale: context.targetLocale,
    hints: context.surroundingText?.sort() // Sort for determinism
  });

  const payload = `${normalizedText}::${contextString}`;
  return createHash('sha256').update(payload).digest('hex');
}

3. Translation Service Implementation

This service handles caching, fallbacks, and LLM invocation.

import { Redis } from 'ioredis';

export class TranslationService {
  private cache: Redis;
  private llmProvider: LLMProvider; // Abstracted LLM client
  private fallbackService: FallbackService;

  constructor(config: ServiceConfig) {
    this.cache = new Redis(config.redisUrl);
    this.llmProvider = new LLMProvider(config.llmApiKey);
    this.fallbackService = new FallbackService();
  }

  async translate(
    text: string, 
    context: TranslationContext
  ): Promise<string> {
    // 1. PII Redaction
    const sanitizedText = this.redactPII(text);

    // 2. Cache Lookup
    const cacheKey = generateCacheKey(sanitizedText, context);
    const cachedResult = await this.cache.get(cacheKey);
    
    if (cachedResult) {
      return this.restorePII(cachedResult, text);
    }

    // 3. Fallback Check (Optional: Static map for critical paths)
    const fallback = this.fallbackService.get(sanitizedText, context.targetLocale);
    if (fallback) {
      await this.cache.set(cacheKey, fallback, 'EX', 3600); // Cache fallback too
      return this.restorePII(fallback, text);
    }

    // 4. LLM Translation
    try {
      const prompt = this.buildPrompt(sanitizedText, context);
      const translation = await this.llmProvider.complete(prompt);
      
      // 5. Cache Write
      await this.cache.set(cacheKey, translation, 'EX', 86400); // 24h TTL

      return this.restorePII(translation, text);
    } catch (error) {
      // 6. Error Handling & Degradation
      console.error('Translation LLM failure:', error);
      return this.handleFailure(text, context);
    }
  }

  private buildPrompt(text: string, context: TranslationContext): string {
    return `
      Translate the following text from ${context.sourceLocale} to ${context.targetLocale}.
      Tone: ${context.tone}
      Domain: ${context.domain}
      ${context.surroundingText ? `Context hints: ${context.surroundingText.join(', ')}` : ''}
      
      Text to translate: "${text}"
      
      Output only the translated text. Do not add explanations.
    `;
  }
}

4. Architecture Decisions

• Edge vs. Centralized: Deploy the cache and routing logic to the Edge (e.g., Cloudflare Workers, Vercel Edge) to reduce latency for cache hits. LLM calls should be centralized or routed to the nearest inference endpoint to manage token costs and security.
• Vector vs. Hash Cache: For high-volume apps with paraphrasing, implement a vector cache (e.g., pgvector, Pinecone) where the key is the embedding of the text+context. This allows matching "Submit" and "Send" if the context is identical. For most apps, a deterministic hash cache provides 80% of the benefit with lower complexity.
• PII Handling: Never send raw user data to LLMs. Implement a redaction layer that replaces PII patterns with tokens before translation and restores them post-translation.

Pitfall Guide

1. Context Collapse

Mistake: Sending isolated strings to the LLM without context. Impact: "Bank" translates to financial institution when the UI refers to a river bank. Best Practice: Always pass surroundingText or explicit domain hints in the prompt. Use UI tree analysis to extract parent labels as context.

2. Cache Collisions

Mistake: Caching translations based only on the source string hash. Impact: "Apple" (fruit) and "Apple" (company) return the same translation. Best Practice: Include context metadata in the cache key generation. Ensure the key reflects domain and tone.

3. Token Blowouts on Dynamic Content

Mistake: Translating large blocks of text containing many variables in a single request. Impact: High latency, potential truncation, and loss of variable integrity. Best Practice: Chunk long texts. Extract variables, translate the template, and re-inject variables. Validate that variable counts match before and after translation.

4. Hallucination of Structure

Mistake: LLM alters HTML tags, markdown syntax, or variable placeholders. Impact: Broken UI, XSS vulnerabilities, or runtime errors. Best Practice: Use system prompts to enforce structure preservation. Implement post-processing validation to check for balanced tags and variable presence.

5. Ignoring Fallback Chains

Mistake: Assuming the LLM API is always available. Impact: Application becomes unusable in target locales during outages. Best Practice: Implement a multi-tier fallback: Semantic Cache → Static Map → Source Text → Error State. Log all fallback usages for monitoring.

6. Cost Leakage from Low-Confidence Caching

Mistake: Using a semantic cache with too low a similarity threshold. Impact: Returning slightly incorrect translations to save cost. Best Practice: Tune the similarity threshold based on evaluation data. For critical UI strings, require exact matches or high thresholds (>0.95). For user-generated content, a lower threshold may be acceptable.

7. Security and Data Residency

Mistake: Sending sensitive data to LLM providers in regions with non-compliant data residency. Impact: GDPR/CCPA violations. Best Practice: Configure LLM routing based on data classification. Use enterprise LLM endpoints with data processing agreements. Redact PII at the edge before transmission.

Production Bundle

Action Checklist

• Define Context Schema: Establish a standardized TranslationContext interface across all services.
• Implement PII Redaction: Deploy a regex/NLP-based redaction layer before LLM calls.
• Deploy Semantic Cache: Integrate Redis or vector DB with cache key generation including context.
• Configure Fallback Chain: Set up static maps and source-text fallbacks for critical paths.
• Add Guardrails: Implement prompt injection protection and output validation for tags/variables.
• Set Up Monitoring: Track cache hit rates, latency percentiles, cost per translation, and fallback frequency.
• Create Evaluation Dataset: Maintain a golden set of translations for automated regression testing.
• Rate Limiting: Apply per-tenant or per-user rate limits to prevent cost abuse.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High Traffic Static UI	Semantic Cache + Static Fallback	95%+ cache hits; minimal latency.	Near zero marginal cost.
User-Generated Content	Raw LLM + Chunking	Content is unique; caching yields low hits.	Linear cost scaling; optimize with smaller models.
Real-Time Chat	Streaming LLM + Edge Cache	Low latency required; context is ephemeral.	Moderate cost; prioritize speed over cache depth.
Regulated Data	On-Prem LLM / Redaction Pipeline	Data cannot leave premises.	High infrastructure cost; zero API fees.
Low Volume / Long Tail	Static i18n + AI on Demand	Insufficient volume to justify cache overhead.	Low cost; pay-per-use only.

Configuration Template

// translation.config.ts
export const TranslationConfig = {
  cache: {
    provider: 'redis',
    url: process.env.REDIS_URL,
    ttl: 86400, // 24 hours
    semanticThreshold: 0.92, // Cosine similarity threshold
  },
  llm: {
    provider: 'openai', // or 'anthropic', 'azure'
    model: 'gpt-4o-mini', // Cost-optimized model
    apiKey: process.env.LLM_API_KEY,
    maxRetries: 3,
    timeout: 2000,
  },
  safety: {
    piiRedaction: true,
    allowedDomains: ['general', 'tech', 'finance'],
    guardrails: {
      enforceTags: true,
      enforceVariables: true,
    },
  },
  fallback: {
    strategy: 'static-map', // 'static-map' | 'source-text'
    mapPath: './locales/fallback.json',
  },
  monitoring: {
    enabled: true,
    metricsPrefix: 'ai.translation',
  },
};

Quick Start Guide

1. Initialize Service:

   npm install ioredis @anthropic-ai/sdk
   

2. Configure Environment:

   export REDIS_URL="redis://localhost:6379"
   export LLM_API_KEY="sk-..."
   

3. Run Translation:

   import { TranslationService } from './TranslationService';
   import { TranslationConfig } from './translation.config';
   
   const service = new TranslationService(TranslationConfig);
   
   const result = await service.translate("Submit", {
     sourceLocale: "en",
     targetLocale: "es",
     tone: "formal",
     domain: "finance",
     surroundingText: ["Credit Card Payment", "Verify Details"]
   });
   
   console.log(result); // "Enviar"
   

4. Verify Cache: Run the same request twice. The second request should complete in <10ms and log a cache hit.
5. Test Fallback: Stop Redis and LLM. Ensure the service returns the source text or static fallback without crashing.

AI-powered translation is a solved problem at the model level but remains a complex engineering challenge at the system level. By prioritizing caching, context injection, and robust fallbacks, you can deliver high-quality localization with performance and cost profiles that compete with traditional i18n.