Rate limiting and throttling

igible when backed by Redis 7+ or equivalent in-memory stores, yet accuracy gains reduce false-positive rejections by up to 60% during traffic anomalies.

Core Solution

Implementing production-grade rate limiting requires algorithmic precision, distributed state consistency, and explicit client communication. The following implementation uses a sliding window counter backed by Redis 7+, written in TypeScript (Node.js 20+), with atomic Lua scripting to prevent race conditions.

Step 1: Define Policy Scope and Identity Resolution

Rate limits must be scoped to identifiable entities: IP address, API key, tenant ID, or authenticated user. Identity resolution should occur before middleware execution to avoid duplicate lookups.

interface RateLimitPolicy {
  identifier: string;
  maxRequests: number;
  windowSeconds: number;
}

Step 2: Atomic Lua Script for Distributed Consistency

Redis executes Lua scripts atomically, eliminating race conditions in multi-node deployments. This script calculates the sliding window count, updates the sorted set, and returns remaining capacity in a single round-trip. The random suffix on ZADD ensures unique scores when multiple requests arrive in the same millisecond.

-- sliding_window.lua
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])

local window_start = now - window
redis.call('ZREMRANGEBYSCORE', key, '-inf', window_start)
local current_count = redis.call('ZCARD', key)

if current_count < limit then
  -- Append random suffix to guarantee unique score in same millisecond
  local member = now .. '-' .. math.random(1000000)
  redis.call('ZADD', key, now, member)
  -- TTL set to 2x window to prevent premature eviction during high load
  redis.call('EXPIRE', key, window * 2)
  return {1, limit - current_count - 1, window}
else
  return {0, 0, window}
end

Step 3: Express Middleware Implementation

This middleware integrates the Lua script, enforces the policy, and returns standard X-RateLimit-* headers. It uses ioredis for connection pooling and script caching.

import { Request, Response, NextFunction } from 'express';
import Redis from 'ioredis';
import { readFileSync } from 'fs';
import { join } from 'path';

const redis = new Redis({
  host: process.env.REDIS_HOST || '127.0.0.1',
  port: parseInt(process.env.REDIS_PORT || '6379'),
  maxRetriesPerRequest: 3,
  retryStrategy: (times) => Math.min(times * 50, 2000),
});

const luaScript = readFileSync(join(__dirname, 'sliding_window.lua'), 'utf8');

export function rateLimitMiddleware(policy: RateLimitPolicy) {
  return async (req: Request, res: Response, next: NextFunction) => {
    const key = `ratelimit:${policy.identifier}:${req.ip}`;
    const now = Date.now() / 1000;

    try {
      const result = await redis.eval(
        luaScript,
        1,
        key,
        now,
        policy.windowSeconds,
        policy.maxRequests
      ) as number[];

      const [allowed, remaining, resetWindow] = result;
      const resetTime = Math.ceil(now + resetWindow);

      res.set('X-RateLimit-Limit', String(policy.maxRequests));
      res.set('X-RateLimit-Remaining', String(remaining));
      res.set('X-RateLimit-Reset', String(resetTime));

      if (allowed === 1) {
        return next();
      }

      res.status(429).json({
        error: 'Too Many Requests',
        retryAfter: resetWindow,
      });
    } catch (err) {
      // Fail-open: allow request if Redis is unreachable to prevent cascading failures
      console.error('Rate limit check failed:', err);
      res.set('X-RateLimit-Remaining', 'unknown');
      return next();
    }
  };
}

Step 4: Integration and Usage

Apply the middleware to specific routes or globally. Scope policies per tenant or endpoint.

import express from 'express';
import { rateLimitMiddleware } from './rateLimitMiddleware';

const app = express();
app.use(express.json());

// Strict limit for payment endpoints
app.post('/api/v1/payments', rateLimitMiddleware({
  identifier: 'payment_api',
  maxRequests: 10,
  windowSeconds: 60,
}), (req, res) => {
  res.json({ status: 'processed' });
});

// Standard limit for public endpoints
app.get('/api/v1/data', rateLimitMiddleware({
  identifier: 'public_api',
  maxRequests: 100,
  windowSeconds: 60,
}), (req, res) => {
  res.json({ data: [] });
});

app.listen(3000, () => console.log('Server running on port 3000'));

Pitfall Guide

Production rate limiting introduces subtle failure modes. Use this troubleshooting matrix to diagnose and resolve common issues:

Symptom	Root Cause	Resolution
Clients receive 429 unexpectedly during normal traffic	Fixed-window boundary spikes or aggressive sliding window drift	Switch to Token Bucket algorithm; tune windowSeconds to align with client retry backoff (recommend 5–10s)
Redis memory usage grows unbounded	Sorted set keys lack proper TTL or EXPIRE drifts	Ensure EXPIRE is set to windowSeconds * 2; run redis-cli --bigkeys weekly; implement key prefix cleanup cron
High latency on /api routes (>50ms added)	EVAL instead of EVALSHA; synchronous Redis calls blocking event loop	Preload Lua script with SCRIPT LOAD; use ioredis pipeline; offload to sidecar (Envoy/NGINX) if latency >20ms
Inconsistent limits across pods	In-memory counters or non-atomic Redis operations	Verify Lua script executes atomically; confirm all pods share the same Redis cluster; disable local fallback in distributed mode
429 responses lack Retry-After header	Middleware doesn't calculate reset time or returns static values	Compute X-RateLimit-Reset as current_time + window_seconds; return Retry-After in seconds for HTTP/1.1 compliance

Debugging Workflow:

1. Enable Redis MONITOR temporarily to trace key patterns: redis-cli monitor | grep ratelimit:
2. Validate Lua script execution time: redis-cli --latency-history
3. Simulate burst traffic with k6:

   import http from 'k6/http';
   export let options = {
     stages: [{ duration: '30s', target: 200 }, { duration: '1m', target: 200 }],
   };
   export default () => http.get('http://localhost:3000/api/v1/data');
   

4. Check middleware placement: Rate limiting must execute before authentication and payload parsing to prevent resource exhaustion on invalid requests.

Production Bundle

Deploying rate limiting requires operational rigor. Follow this checklist to ensure stability, observability, and maintainability.

Deployment Checklist

• Redis 7+ cluster with AOF persistence enabled (appendonly yes)
• Lua script preloaded via SCRIPT LOAD during application startup
• Middleware positioned before body parsers and authentication routes
• X-RateLimit-* headers standardized across all endpoints
• Fail-open policy configured for Redis outages (log alert, allow traffic)
• Policy configuration externalized to environment variables or config service

Monitoring & Alerting Instrument the following metrics using Prometheus/OpenTelemetry:

• rate_limit_rejected_total{endpoint, tenant}: Track rejection rates per scope
• rate_limit_latency_ms: P95 latency added by middleware
• redis_connections_active: Monitor connection pool exhaustion
• Alert thresholds: Reject rate >5% sustained for 2m; Latency P95 >30ms; Redis memory >80%

Testing Strategy

• Unit Tests: Mock Redis responses to validate boundary conditions (exact limit, limit+1, window reset)
• Integration Tests: Spin up local Redis 7 via Docker; run k6 burst simulations; verify header accuracy
• Chaos Tests: Kill Redis leader node; verify fail-open behavior and automatic reconnection
• Policy Validation: Use schema validation to prevent maxRequests: 0 or windowSeconds: <1 in CI/CD

Operational Runbook

• Updating Policies: Never restart services to change limits. Load policies from a dynamic config store (Consul/Vault) and hot-reload middleware.
• Handling False Positives: Whitelist internal service accounts by identifier prefix (svc-); implement exponential backoff guidance in 429 responses.
• Scaling Redis: When key count exceeds 10M or P99 latency >10ms, shard by tenant ID using consistent hashing; migrate to Redis Cluster mode.
• Decommissioning: Add X-RateLimit-Status: dry-run header to log rejections without blocking traffic before enforcing new policies.