Circuit breaker implementation

ble thresholds, timers, and metrics hooks.

Step 1: Define Configuration and State Types

export type CircuitState = 'CLOSED' | 'OPEN' | 'HALF_OPEN';

export interface CircuitBreakerConfig {
  failureThreshold: number;       // Errors to trip the circuit
  successThreshold: number;       // Successes to close from HALF_OPEN
  timeoutMs: number;              // Duration to wait before HALF_OPEN
  halfOpenMaxRequests: number;    // Max concurrent requests in HALF_OPEN
  onError?: (error: Error) => void;
  onStateChange?: (from: CircuitState, to: CircuitState) => void;
}

Step 2: Implement the State Machine

export class CircuitBreaker<T> {
  private state: CircuitState = 'CLOSED';
  private failureCount = 0;
  private successCount = 0;
  private halfOpenRequests = 0;
  private openTimer: NodeJS.Timeout | null = null;

  constructor(
    private readonly config: CircuitBreakerConfig,
    private readonly fallback?: () => Promise<T>
  ) {}

  getState(): CircuitState {
    return this.state;
  }

  private transition(newState: CircuitState): void {
    const previous = this.state;
    this.state = newState;
    this.config.onStateChange?.(previous, newState);
    
    if (newState === 'OPEN') {
      this.failureCount = 0;
      this.successCount = 0;
      this.halfOpenRequests = 0;
      this.openTimer = setTimeout(() => this.enterHalfOpen(), this.config.timeoutMs);
    } else if (newState === 'CLOSED') {
      this.failureCount = 0;
      this.successCount = 0;
    }
  }

  private enterHalfOpen(): void {
    this.transition('HALF_OPEN');
  }

  private recordSuccess(): void {
    if (this.state === 'HALF_OPEN') {
      this.successCount++;
      this.halfOpenRequests--;
      if (this.successCount >= this.config.successThreshold) {
        this.transition('CLOSED');
      }
    } else if (this.state === 'CLOSED') {
      this.failureCount = Math.max(0, this.failureCount - 1);
    }
  }

  private recordFailure(): void {
    if (this.state === 'HALF_OPEN') {
      this.halfOpenRequests--;
      this.transition('OPEN');
    } else if (this.state === 'CLOSED') {
      this.failureCount++;
      if (this.failureCount >= this.config.failureThreshold) {
        this.transition('OPEN');
      }
    }
  }

Step 3: Async Execution with Fallback and Rate Limiting

  async execute(fn: () => Promise<T>): Promise<T> {
    if (this.state === 'OPEN') {
      return this.handleFallback();
    }

    if (this.state === 'HALF_OPEN' && this.halfOpenRequests >= this.config.halfOpenMaxRequests) {
      return this.handleFallback();
    }

    if (this.state === 'HALF_OPEN') {
      this.halfOpenRequests++;
    }

    try {
      const result = await fn();
      this.recordSuccess();
      return result;
    } catch (error) {
      this.recordFailure();
      this.config.onError?.(error as Error);
      return this.handleFallback();
    }
  }

  private async handleFallback(): Promise<T> {
    if (this.fallback) {
      return this.fallback();
    }
    throw new Error(`Circuit breaker is ${this.state} and no fallback provided`);
  }

Step 4: Architecture Decisions and Rationale

1. State Machine over Decorator: A class-based state machine isolates state transitions from execution logic. This prevents race conditions where concurrent failures incorrectly trip or reset the circuit.
2. Half-Open Request Limiting: The halfOpenMaxRequests guard ensures only a controlled subset of traffic probes the dependency. Unbounded probing in HALF_OPEN state re-trips the circuit immediately and masks recovery signals.
3. Failure Classification: The implementation counts all thrown errors as failures. In production, filter by error type (e.g., TimeoutError, ServerError) before calling recordFailure(). Client errors (4xx) should not trip the breaker.
4. Fallback as First-Class Citizen: Fallbacks are executed synchronously when OPEN or HALF_OPEN is saturated. This preserves client responsiveness and prevents cascading timeouts. Fallbacks should be idempotent and stateless.
5. Metrics Hooks: onError and onStateChange callbacks integrate with Prometheus, OpenTelemetry, or Datadog. Track state transitions, failure velocity, and fallback invocation rates to tune thresholds dynamically.
6. Async Safety: The implementation avoids synchronous blocking. All state mutations occur within async boundaries, preventing event loop starvation in Node.js/TypeScript runtimes.

Pitfall Guide

1. Treating HALF_OPEN as Fully Open Mistake: Allowing all queued requests to execute when transitioning to HALF_OPEN. Impact: Immediate re-tripping, false recovery signals, and amplified downstream load. Best Practice: Enforce halfOpenMaxRequests strictly. Use a single probe request or a small fixed window. Only transition to CLOSED after sustained success.

2. Misaligned Failure Thresholds Mistake: Using absolute failure counts without considering request volume or time window. Impact: High-traffic services trip instantly; low-traffic services never trip. Best Practice: Normalize thresholds by request rate. Implement rolling windows (e.g., 10 failures in 60 seconds) rather than cumulative counters. Adjust based on service SLOs.

3. Missing or Heavy Fallbacks Mistake: Omitting fallbacks or implementing synchronous, CPU-bound fallbacks. Impact: Clients receive hard failures or experience latency spikes during OPEN state. Best Practice: Provide lightweight, cached, or degraded fallbacks. Ensure fallbacks execute in <10ms and avoid external I/O. Log fallback invocations for observability.

4. Blocking the Event Loop Mistake: Using setTimeout synchronously or blocking state transitions with heavy computation. Impact: Event loop starvation, increased P99 latency, and false circuit trips. Best Practice: Keep state mutations O(1). Offload metrics aggregation to background workers. Use setImmediate or async queues if processing fails.

5. Stateless Deployment in Clusters Mistake: Deploying circuit breakers without shared state across instances. Impact: Inconsistent tripping, uneven traffic distribution, and partial recovery. Best Practice: Use consistent hashing or service mesh routing to pin requests to specific instances. Alternatively, share state via Redis or a control plane for global circuit awareness.

6. Mixing Retries Inside Circuit Breakers Mistake: Wrapping retries within the execute() function. Impact: Retries consume HALF_OPEN probes, mask true failure rates, and delay recovery. Best Practice: Separate retry logic from circuit breaker logic. Retries should only apply to transient errors in CLOSED state. The circuit breaker should see the final outcome, not intermediate attempts.

7. Ignoring Timeout vs Failure Distinction Mistake: Treating timeouts and application errors identically. Impact: Over-tripping on network latency, under-tripping on logic bugs. Best Practice: Classify errors before recording. Timeouts should increment a separate latency metric and trigger faster HALF_OPEN probing. Application errors should follow standard failure thresholds.

Production Bundle

Action Checklist

• Define failure thresholds based on rolling windows, not cumulative counts
• Implement HALF_OPEN request limiting to prevent probe amplification
• Attach lightweight fallbacks for every protected dependency
• Separate retry logic from circuit breaker execution boundaries
• Instrument state transitions and fallback invocations with metrics
• Validate error classification before recording failures (exclude 4xx)
• Test HALF_OPEN recovery paths under simulated degradation
• Align timeout values with downstream SLOs and circuit breaker timeout

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High-throughput public API	Rolling window thresholds + fallback cache	Prevents instant tripping, maintains availability	Low compute, moderate cache cost
Internal microservice mesh	Shared state circuit breaker via service mesh	Consistent tripping across instances, reduces blast radius	Medium infrastructure, lower opex
Batch/async job pipeline	Delayed HALF_OPEN probing + strict failure classification	Avoids false recovery, aligns with job retry policies	Low runtime cost, higher monitoring overhead
Legacy monolith migration	Basic breaker + synchronous fallback	Fast deployment, minimal refactoring	Low initial cost, higher technical debt

Configuration Template

import { CircuitBreaker, CircuitBreakerConfig } from './circuit-breaker';

const config: CircuitBreakerConfig = {
  failureThreshold: 5,
  successThreshold: 3,
  timeoutMs: 30000,
  halfOpenMaxRequests: 1,
  onError: (err) => console.error(`[CB] Failure recorded: ${err.message}`),
  onStateChange: (from, to) => console.log(`[CB] State: ${from} -> ${to}`),
};

const paymentCircuit = new CircuitBreaker<PaymentResponse>(config, async () => {
  // Fallback: return cached response or queue for later
  return { status: 'queued', id: generateId() };
});

// Usage
const response = await paymentCircuit.execute(() => 
  fetchPaymentGateway(payload)
);

Quick Start Guide

1. Install the implementation or integrate the class into your service layer.
2. Define a CircuitBreakerConfig matching your dependency's SLO and traffic profile.
3. Wrap downstream calls with circuit.execute(async () => await dependencyCall()).
4. Attach a lightweight fallback that returns degraded but valid responses.
5. Deploy with metrics hooks enabled and monitor HALF_OPEN transitions for 24 hours before tuning thresholds.