API timeout configuration

Implementing production-grade timeout configuration requires a defense-in-depth strategy that separates concerns across client, gateway, and infrastructure layers. The following implementation demonstrates a TypeScript-based approach using native fetch and AbortController, structured for extensibility and observability.

Step 1: Classify Endpoints by Criticality and Expected Latency

Map your outbound calls into tiers:

• Tier 1 (Critical): Payment processing, authentication, checkout. Tolerates higher latency, requires retry budget, strict circuit breaker integration.
• Tier 2 (Standard): Product catalog, user profiles, search. Moderate latency tolerance, standard retry, fallback caching.
• Tier 3 (Non-Critical): Analytics, logging, telemetry, recommendations. Strict timeouts, no retry, fire-and-forget acceptable.

Step 2: Implement Multi-Phase Timeout Configuration

HTTP requests fail at different stages. Configure each phase explicitly:

export interface TimeoutConfig {
  // Total wall-clock limit for the entire request lifecycle
  overall: number;
  // Time to establish TCP connection
  connect: number;
  // Time for TLS handshake (if applicable)
  tls: number;
  // Time to send request body
  write: number;
  // Time to wait for first byte of response
  read: number;
}

Step 3: Build a Resilient Fetch Wrapper

import { setTimeout as sleep } from 'timers/promises';

export class ResilientApiClient {
  private readonly baseUrl: string;
  private readonly defaultTimeouts: Record<string, TimeoutConfig>;

  constructor(baseUrl: string, timeouts: Record<string, TimeoutConfig>) {
    this.baseUrl = baseUrl;
    this.defaultTimeouts = timeouts;
  }

  async request<T>(
    path: string,
    tier: 'critical' | 'standard' | 'non-critical',
    options: RequestInit = {}
  ): Promise<T> {
    const config = this.defaultTimeouts[tier];
    const controller = new AbortController();
    const timers: NodeJS.Timeout[] = [];

    // Phase-specific abort triggers
    const start = Date.now();
    const phaseAbort = (phase: string, limit: number) => {
      const timer = setTimeout(() => {
        controller.abort(new Error(`Timeout: ${phase} exceeded ${limit}ms`));
      }, limit);
      timers.push(timer);
    };

    phaseAbort('connect', config.connect);
    phaseAbort('tls', config.tls);
    phaseAbort('write', config.write);
    phaseAbort('read', config.read);
    phaseAbort('overall', config.overall);

    try {
      const response = await fetch(`${this.baseUrl}${path}`, {
        ...options,
        signal: controller.signal,
      });

      // Clear all timers on successful response start
      timers.forEach(clearTimeout);

      if (!response.ok) {
        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
      }

      return await response.json();
    } catch (error) {
      timers.forEach(clearTimeout);
      // Attach phase metadata for observability
      if (error instanceof Error && error.name === 'AbortError') {
        const duration = Date.now() - start;
        throw new TimeoutError(error.message, tier, duration);
      }
      throw error;
    }
  }
}

export class TimeoutError extends Error {
  constructor(
    message: string,
    public readonly tier: string,
    public readonly durationMs: number
  ) {
    super(message);
    this.name = 'TimeoutError';
  }
}

Step 4: Configure Tiered Timeouts

const TIMEOUT_PRESETS: Record<string, TimeoutConfig> = {
  critical: {
    overall: 8000,
    connect: 1500,
    tls: 2000,
    write: 1000,
    read: 3500,
  },
  standard: {
    overall: 3000,
    connect: 800,
    tls: 1000,
    write: 500,
    read: 700,
  },
  'non-critical': {
    overall: 1500,
    connect: 500,
    tls: 600,
    write: 200,
    read: 200,
  },
};

Step 5: Integrate with Retry and Circuit Breaker

Timeouts must be paired with backpressure mechanisms. A failed timeout should not trigger immediate retry without exponential backoff and jitter. Implement a circuit breaker that opens when timeout error rate exceeds 15% over a 10-second window, preventing further resource allocation to degraded downstreams.

Architecture Decisions and Rationale

• Separation of phase timeouts: Network congestion, DNS failures, and slow backend processing manifest at different stages. Uniform timeouts mask root causes and waste client resources waiting for phases that have already failed.
• Tiered configuration: Business logic dictates tolerance. Charging a user's credit card warrants longer wait times than fetching a recommendation widget. Aligning timeouts to value prevents resource starvation on non-critical paths.
• AbortController over legacy timeouts: Native setTimeout with request cancellation ensures deterministic cleanup. Frameworks that rely on internal timers often leak sockets or fail to release connection pool slots.
• Gateway alignment: Client timeouts must be strictly less than gateway/load balancer timeouts. If the gateway drops the connection at 60s, but the client waits 90s, the client experiences a TCP RST that is harder to handle gracefully than a controlled abort.

Pitfall Guide

1. Relying on Framework Defaults

Default timeout values are optimized for developer convenience, not production resilience. Node.js's 120-second default allows a single degraded endpoint to hold open hundreds of connections, exhausting file descriptors and memory. Always override defaults explicitly.

2. Applying a Single Timeout Value Across All Requests

Uniform timeouts ignore endpoint criticality and downstream capacity. A 5-second timeout for a payment service is too aggressive; a 5-second timeout for analytics is too permissive. Tiered configuration prevents resource misallocation.

3. Ignoring Connection and TLS Timeouts

Many teams only configure read/overall timeouts. DNS resolution failures, TCP handshake delays, or TLS certificate validation hangs can block threads for seconds before the request even reaches the server. Explicit connect and TLS timeouts catch infrastructure-level failures early.

4. Misaligning Client and Gateway Timeouts

If your load balancer has a 60-second timeout and your client waits 90 seconds, the client receives a TCP reset instead of a clean HTTP response. Client timeouts must always be shorter than infrastructure timeouts to ensure predictable error handling.

5. Retrying Without Exponential Backoff and Jitter

Timeout failures often indicate transient congestion. Immediate retry amplifies load on an already struggling downstream. Implement exponential backoff (e.g., 100ms → 200ms → 400ms) with random jitter (±25%) to prevent thundering herd effects.

6. Treating Timeouts as Errors Instead of Flow Control

Timeouts are not failures; they are circuit breakers. Logging every timeout as an ERROR floods monitoring systems and obscures real incidents. Classify timeouts as WARN or INFO, aggregate by tier and phase, and trigger circuit breakers based on rate thresholds.

7. No Timeout Phase Observability

Without instrumentation that tracks which phase aborted the request, teams cannot distinguish between slow backends, network issues, or misconfigured gateways. Emit structured metrics: timeout_phase, tier, duration_ms, downstream_host.

Best Practices from Production:

• Enforce timeout configuration through linting or framework middleware. Reject requests without explicit timeout objects.
• Use connection pool metrics to validate timeout effectiveness. Pool utilization should stabilize under load, not climb monotonically.
• Align timeout budgets with retry budgets. If overall timeout is 3s and max retries is 2, each attempt should target ≤1s to leave headroom.
• Document timeout expectations in API contracts. Downstream services should publish SLOs that inform client timeout configuration.

Production Bundle

Action Checklist

• Map all outbound API calls to criticality tiers (critical, standard, non-critical)
• Define explicit timeout values for connect, tls, write, read, and overall phases per tier
• Ensure client timeouts are strictly shorter than gateway/load balancer timeouts
• Implement AbortController-based cancellation with phase-level tracking
• Integrate exponential backoff with jitter for retry logic
• Configure circuit breaker thresholds based on timeout error rates (e.g., 15% over 10s)
• Emit structured observability metrics for timeout phase, tier, and duration
• Validate configuration under load using chaos testing (inject latency, drop packets, simulate DNS failures)

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Payment/Checkout APIs	Tiered: overall 8s, read 3.5s, retry with backoff	High business value justifies extended wait times; prevents false declines	Higher compute per request, but reduces revenue loss
Internal Microservice Calls	Tiered: overall 2s, strict connect/read limits	Low latency expectation; fast failure enables circuit breaker activation	Lower resource consumption, improved system stability
Third-Party SaaS Integrations	Tiered: overall 5s, gateway-aligned, circuit breaker mandatory	Unpredictable downstream behavior requires strict isolation	Prevents cascade failures, minimal infra cost
Analytics/Telemetry Endpoints	Tiered: overall 1.5s, no retry, fire-and-forget fallback	Non-critical data; timeout is acceptable failure mode	Near-zero impact on core system resources
Legacy Monolith Dependencies	Tiered: overall 10s, connection pooling, async offload	Slow legacy systems require extended windows but must not block modern services	Higher timeout budget, but isolated via async queues

Configuration Template

// timeout.config.ts
export const TIMEOUT_CONFIG = {
  critical: {
    overall: 8000,
    connect: 1500,
    tls: 2000,
    write: 1000,
    read: 3500,
    retries: 2,
    backoffBase: 200,
    circuitBreakerThreshold: 0.15,
    circuitBreakerWindowMs: 10000,
  },
  standard: {
    overall: 3000,
    connect: 800,
    tls: 1000,
    write: 500,
    read: 700,
    retries: 1,
    backoffBase: 100,
    circuitBreakerThreshold: 0.2,
    circuitBreakerWindowMs: 10000,
  },
  'non-critical': {
    overall: 1500,
    connect: 500,
    tls: 600,
    write: 200,
    read: 200,
    retries: 0,
    backoffBase: 0,
    circuitBreakerThreshold: 0.25,
    circuitBreakerWindowMs: 5000,
  },
};

// Usage example
const client = new ResilientApiClient('https://api.internal', TIMEOUT_CONFIG);

try {
  const order = await client.request('/v1/orders', 'critical', { method: 'POST' });
  console.log('Order created:', order);
} catch (err) {
  if (err instanceof TimeoutError) {
    console.warn(`Timeout at ${err.tier} tier after ${err.durationMs}ms`);
    // Trigger fallback or circuit breaker logic
  }
}

Quick Start Guide

1. Install dependencies: Ensure your runtime supports fetch and AbortController (Node.js 18+, modern browsers, or polyfill via undici).
2. Create timeout presets: Copy the TIMEOUT_CONFIG template and adjust values to match your downstream SLOs and business criticality.
3. Instantiate the client: Initialize ResilientApiClient with your base URL and timeout presets. Replace existing fetch or HTTP client calls with client.request(path, tier, options).
4. Add observability: Hook into the TimeoutError catch block to emit metrics to your monitoring stack (Prometheus, Datadog, CloudWatch). Track timeout_phase, tier, and duration_ms.
5. Validate under load: Run a synthetic load test injecting 200ms-2s latency spikes. Verify that connection pool saturation stays below 50%, p99 latency remains stable, and circuit breakers trigger at configured thresholds.