/etc/nginx/conf.d/api.conf

| 24% | High complexity, low cacheability |

Key Insight: Full negotiation reduces payload size by 52% and improves cache efficiency by 177% versus static JSON. Latency gains stem from reduced serialization overhead and HTTP/2 multiplexing efficiency when compressed representations align with client Accept-Encoding preferences. Unlike GraphQL, negotiation remains fully cacheable at the edge when Vary headers are correctly scoped.

Core Solution

Prerequisites: Node.js 22 LTS, Express 4.21, Nginx 1.26, accepts@1.3.8, compression@1.7.5.

Step 1: Express Application with RFC-Compliant Negotiation

// server.js
const express = require('express');
const accepts = require('accepts');
const compression = require('compression');

const app = express();
const PORT = process.env.PORT || 3000;

// Align compression with client Accept-Encoding preferences
app.use(compression({
  threshold: 1024,
  filter: (req, res) => {
    if (req.headers['x-no-compression']) return false;
    return compression.filter(req, res);
  }
}));

const USERS = [
  { id: 1, name: 'Alice', email: 'alice@example.com', role: 'admin', metadata: { lastLogin: '2024-01-15' } },
  { id: 2, name: 'Bob', email: 'bob@example.com', role: 'user', metadata: { lastLogin: '2024-02-20' } }
];

app.get('/api/users', (req, res) => {
  const accept = accepts(req);

  // 1. Parse Accept header with proper q-value resolution
  const type = accept.types(['application/json', 'application/xml', 'text/csv']);
  
  if (!type) {
    return res.status(406).json({ 
      error: 'Not Acceptable', 
      supported: ['application/json', 'application/xml', 'text/csv'] 
    });
  }

  // 2. Set Vary header to prevent cache poisoning
  res.vary('Accept');
  res.vary('Accept-Encoding');

  // 3. Serialize based on negotiated type
  switch (type) {
    case 'application/xml':
      res.set('Content-Type', 'application/xml');
      res.send(`<users>${USERS.map(u => `<user id="${u.id}"><name>${u.name}</name></user>`).join('')}</users>`);
      break;
    case 'text/csv':
      res.set('Content-Type', 'text/csv');
      res.send('id,name\n' + USERS.map(u => `${u.id},${u.name}`).join('\n'));
      break;
    default: // application/json
      res.set('Content-Type', 'application/json');
      res.json(USERS);
  }
});

app.listen(PORT, () => console.log(`Server running on port ${PORT}`));

Step 2: Nginx Reverse Proxy & Cache Configuration

# /etc/nginx/conf.d/api.conf
proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=60m;

server {
    listen 80;
    server_name api.example.com;

    location /api/ {
        proxy_pass http://127.0.0.1:3000;
        proxy_cache api_cache;
        
        # Critical: Include negotiation headers in cache key to prevent fragmentation
        proxy_cache_key "$scheme$request_method$host$request_uri$http_accept$http_accept_encoding";
        
        # Pass Vary headers from upstream to edge
        proxy_cache_valid 200 10m;
        add_header X-Cache-Status $upstream_cache_status always;
    }
}

Step 3: Client Verification

# Request JSON with gzip
curl -v -H "Accept: application/json;q=0.9, text/csv;q=0.5" \
       -H "Accept-Encoding: gzip, br" \
       http://localhost/api/users

# Request XML (triggers 406 if unsupported, or returns XML)
curl -v -H "Accept: application/xml" http://localhost/api/users

# Verify cache status on subsequent requests
curl -I -H "Accept: application/json" http://localhost/api/users | grep X-Cache-Status

Pitfall Guide

Symptom	Root Cause	Resolution
406 Not Acceptable on valid requests	Naive string matching or missing accepts library	Replace req.headers.accept.includes() with accepts(req).types(). Verify client sends valid MIME types.
Cache hit rate drops to <20%	Vary header includes too many headers (e.g., User-Agent, Accept-Language)	Scope Vary strictly to Accept and Accept-Encoding. Use res.vary() programmatically instead of static headers.
Compression not applied	compression middleware threshold too high or Accept-Encoding mismatch	Set threshold: 1024. Verify client sends Accept-Encoding: gzip, br. Check Nginx gzip vs app-level compression conflicts.
CDN serves wrong format	CDN ignores Vary or cache key excludes negotiation headers	Map Vary to CDN cache key rules. In Nginx, explicitly include $http_accept in proxy_cache_key.
Stale responses after API update	Vary mismatch between app and proxy	Ensure both Express and Nginx use identical Vary header sets. Run curl -I to verify Vary: Accept, Accept-Encoding in response.

Debugging Commands:

# Inspect actual cache key generated by Nginx
nginx -T | grep proxy_cache_key
curl -s -D - -o /dev/null http://localhost/api/users -H "Accept: application/json"

# Verify Vary header propagation
curl -I -H "Accept: application/json" http://localhost/api/users | grep -i vary

# Test compression negotiation
curl -s -H "Accept-Encoding: gzip" -o - http://localhost/api/users | file -

Production Bundle

Deployment Checklist:

• Node.js 22 LTS + Express 4.21 deployed with NODE_ENV=production
• accepts and compression middleware verified in request lifecycle
• Nginx proxy_cache_key explicitly includes $http_accept and $http_accept_encoding
• Vary header scope limited to Accept, Accept-Encoding (no Accept-Language or User-Agent)
• CDN cache rules mapped to match Nginx proxy_cache_key structure
• Health check endpoint /api/health excluded from negotiation to avoid cache pollution

Monitoring & Alerting:

• http_406_responses_total: Alert if >0.1% of requests (indicates client/library misconfiguration)
• cache_hit_ratio: Alert if drops below 75% (indicates Vary cardinality explosion)
• compression_ratio_avg: Track payload reduction vs uncompressed baseline
• p99_latency_ms: Correlate with negotiation success rate

Load Testing Script (wrk):

wrk -t4 -c100 -d30s -H "Accept: application/json;q=0.8, text/csv;q=0.2" -H "Accept-Encoding: gzip" http://localhost/api/users

Rollback Strategy: If negotiation causes unexpected cache fragmentation or 406 spikes, disable dynamic Vary scoping by reverting to static Content-Type: application/json responses. Nginx cache can be purged via proxy_cache_purge or CDN dashboard. Application fallback: set FORCE_JSON=true env var to bypass accepts() parsing and return JSON unconditionally.