Rate limiting and throttling

interface RateLimitPolicy {
  identifier: string;
  maxRequests: number;
  windowSeconds: number;
}

-- sliding_window.lua
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])

local window_start = now - window
redis.call('ZREMRANGEBYSCORE', key, '-inf', window_start)
local current_count = redis.call('ZCARD', key)

if current_count < limit then
  -- Append random suffix to guarantee unique score in same millisecond
  local member = now .. '-' .. math.random(1000000)
  redis.call('ZADD', key, n

ow, member) -- TTL set to 2x window to prevent premature eviction during high load redis.call('EXPIRE', key, window * 2) return {1, limit - current_count - 1, window} else return {0, 0, window} end

### Step 3: Express Middleware Implementation
This middleware integrates the Lua script, enforces the policy, and returns standard `X-RateLimit-*` headers. It uses `ioredis` for connection pooling and script caching.

```typescript
import { Request, Response, NextFunction } from 'express';
import Redis from 'ioredis';
import { readFileSync } from 'fs';
import { join } from 'path';

const redis = new Redis({
  host: process.env.REDIS_HOST || '127.0.0.1',
  port: parseInt(process.env.REDIS_PORT || '6379'),
  maxRetriesPerRequest: 3,
  retryStrategy: (times) => Math.min(times * 50, 2000),
});

const luaScript = readFileSync(join(__dirname, 'sliding_window.lua'), 'utf8');

export function rateLimitMiddleware(policy: RateLimitPolicy) {
  return async (req: Request, res: Response, next: NextFunction) => {
    const key = `ratelimit:${policy.identifier}:${req.ip}`;
    const now = Date.now() / 1000;

    try {
      const result = await redis.eval(
        luaScript,
        1,
        key,
        now,
        policy.windowSeconds,
        policy.maxRequests
      ) as number[];

      const [allowed, remaining, resetWindow] = result;
      const resetTime = Math.ceil(now + resetWindow);

      res.set('X-RateLimit-Limit', String(policy.maxRequests));
      res.set('X-RateLimit-Remaining', String(remaining));
      res.set('X-RateLimit-Reset', String(resetTime));

      if (allowed === 1) {
        return next();
      }

      res.status(429).json({
        error: 'Too Many Requests',
        retryAfter: resetWindow,
      });
    } catch (err) {
      // Fail-open: allow request if Redis is unreachable to prevent cascading failures
      console.error('Rate limit check failed:', err);
      res.set('X-RateLimit-Remaining', 'unknown');
      return next();
    }
  };
}

Step 4: Integration and Usage

Apply the middleware to specific routes or globally. Scope policies per tenant or endpoint.

import express from 'express';
import { rateLimitMiddleware } from './rateLimitMiddleware';

const app = express();
app.use(express.json());

// Strict limit for payment endpoints
app.post('/api/v1/payments', rateLimitMiddleware({
  identifier: 'payment_api',
  maxRequests: 10,
  windowSeconds: 60,
}), (req, res) => {
  res.json({ status: 'processed' });
});

// Standard limit for public endpoints
app.get('/api/v1/data', rateLimitMiddleware({
  identifier: 'public_api',
  maxRequests: 100,
  windowSeconds: 60,
}), (req, res) => {
  res.json({ data: [] });
});

app.listen(3000, () => console.log('Server running on port 3000'));

Pitfall Guide

Production rate limiting introduces subtle failure modes. Use this troubleshooting matrix to diagnose and resolve common issues:

Symptom	Root Cause	Resolution
Clients receive 429 unexpectedly during normal traffic	Fixed-window boundary spikes or aggressive sliding window drift	Switch to Token Bucket algorithm; tune windowSeconds to align with client retry backoff (recommend 5–10s)
Redis memory usage grows unbounded	Sorted set keys lack proper TTL or EXPIRE drifts	Ensure EXPIRE is set to windowSeconds * 2; run redis-cli --bigkeys weekly; implement key prefix cleanup cron
High latency on /api routes (>50ms added)	EVAL instead of EVALSHA; synchronous Redis calls blocking event loop	Preload Lua script with SCRIPT LOAD; use ioredis pipeline; offload to sidecar (Envoy/NGINX) if latency >20ms
Inconsistent limits across pods	In-memory counters or non-atomic Redis operations	Verify Lua script executes atomically; confirm all pods share the same Redis cluster; disable local fallback in distributed mode
429 responses lack Retry-After header	Middleware doesn't calculate reset time or returns static values	Compute X-RateLimit-Reset as current_time + window_seconds; return Retry-After in seconds for HTTP/1.1 compliance

Debugging Workflow:

1. Enable Redis MONITOR temporarily to trace key patterns: redis-cli monitor | grep ratelimit:
2. Validate Lua script execution time: redis-cli --latency-history
3. Simulate burst traffic with k6:

   import http from 'k6/http';
   export let options = {
     stages: [{ duration: '30s', target: 200 }, { duration: '1m', target: 200 }],
   };
   export default () => http.get('http://localhost:3000/api/v1/data');
   

4. Check middleware placement: Rate limiting must execute before authentication and payload parsing to prevent resource exhaustion on invalid requests.

Production Bundle

Deploying rate limiting requires operational rigor. Follow this checklist to ensure stability, observability, and maintainability.

Deployment Checklist

• Redis 7+ cluster with AOF persistence enabled (appendonly yes)
• Lua script preloaded via SCRIPT LOAD during application startup
• Middleware positioned before body parsers and authentication routes
• X-RateLimit-* headers standardized across all endpoints
• Fail-open policy configured for Redis outages (log alert, allow traffic)
• Policy configuration externalized to environment variables or config service

Monitoring & Alerting Instrument the following metrics using Prometheus/OpenTelemetry:

• rate_limit_rejected_total{endpoint, tenant}: Track rejection rates per scope
• rate_limit_latency_ms: P95 latency added by middleware
• redis_connections_active: Monitor connection pool exhaustion
• Alert thresholds: Reject rate >5% sustained for 2m; Latency P95 >30ms; Redis memory >80%

Testing Strategy

• Unit Tests: Mock Redis responses to validate boundary conditions (exact limit, limit+1, window reset)
• Integration Tests: Spin up local Redis 7 via Docker; run k6 burst simulations; verify header accuracy
• Chaos Tests: Kill Redis leader node; verify fail-open behavior and automatic reconnection
• Policy Validation: Use schema validation to prevent maxRequests: 0 or windowSeconds: <1 in CI/CD

Operational Runbook

• Updating Policies: Never restart services to change limits. Load policies from a dynamic config store (Consul/Vault) and hot-reload middleware.
• Handling False Positives: Whitelist internal service accounts by identifier prefix (svc-); implement exponential backoff guidance in 429 responses.
• Scaling Redis: When key count exceeds 10M or P99 latency >10ms, shard by tenant ID using consistent hashing; migrate to Redis Cluster mode.
• Decommissioning: Add X-RateLimit-Status: dry-run header to log rejections without blocking traffic before enforcing new policies.