Caching strategies for backend

distributed coordination, versioned keys, and stale-while-revalidate fallbacks**. This approach balances flexibility, observability, and failure isolation while keeping the primary datastore authoritative.

Step 1: Architecture Decisions

• Storage Engine: Redis is preferred over in-memory or Memcached due to persistence options, pub/sub invalidation, Lua scripting, and native data structures. For multi-region deployments, use Redis Cluster or managed offerings with cross-region replication.
• Connection Topology: Separate read and write clients. Read clients use replica nodes; write clients target primary nodes. This isolates cache miss storms from write throughput.
• Serialization: JSON is acceptable for simple payloads. For high-throughput systems, switch to MessagePack or Protocol Buffers to reduce serialization overhead by 40–60%.
• Key Design: Use namespaced, versioned keys: v1:service:entity:{id}:{hash}. Versioning enables atomic invalidation without scanning.

Step 2: TypeScript Implementation

The following service implements cache-aside with mutex locking, TTL jitter, and stale-while-revalidate semantics using ioredis.

import Redis from 'ioredis';
import { createHash } from 'crypto';

interface CacheConfig {
  host: string;
  port: number;
  password?: string;
  keyPrefix: string;
  defaultTTL: number; // seconds
  maxRetries: number;
}

interface CacheOptions {
  ttl?: number;
  version?: string;
  staleWhileRevalidate?: boolean;
}

export class CacheService {
  private client: Redis;
  private lockClient: Redis;
  private readonly keyPrefix: string;
  private readonly defaultTTL: number;

  constructor(config: CacheConfig) {
    this.keyPrefix = config.keyPrefix;
    this.defaultTTL = config.defaultTTL;

    const baseOptions = {
      host: config.host,
      port: config.port,
      password: config.password,
      retryStrategy: (times: number) => Math.min(times * 50, 2000),
      maxRetriesPerRequest: config.maxRetries,
      enableReadyCheck: true,
      connectTimeout: 5000,
    };

    this.client = new Redis({ ...baseOptions, enableOfflineQueue: false });
    this.lockClient = new Redis({ ...baseOptions, enableOfflineQueue: false });
  }

  private buildKey(resource: string, id: string, version?: string): string {
    const ver = version || 'v1';
    return `${this.keyPrefix}:${ver}:${resource}:${id}`;
  }

  private applyJitter(ttl: number): number {
    const jitter = Math.floor(Math.random() * ttl * 0.2);
    return ttl - jitter;
  }

  async get<T>(resource: string, id: string, opts?: CacheOptions): Promise<T | null> {
    const key = this.buildKey(resource, id, opts?.version);
    const raw = await this.client.get(key);
    if (!raw) return null;
    return JSON.parse(raw) as T;
  }

  async set<T>(resource: string, id: string, value: T, opts?: CacheOptions): Promise<void> {
    const key = this.buildKey(resource, id, opts?.version);
    const ttl = this.applyJitter(opts?.ttl ?? this.defaultTTL);
    const serialized = JSON.stringify(value);
    await this.client.set(key, serialized, 'EX', ttl);
  }

  async invalidate(resource: string, id: string, version?: string): Promise<void> {
    const key = this.buildKey(resource, id, version);
    await this.client.del(key);
  }

  async getOrSet<T>(
    resource: string,
    id: string,
    fetchFn: () => Promise<T>,
    opts?: CacheOptions
  ): Promise<T> {
    const key = this.buildKey(resource, id, opts?.version);
    const cached = await this.client.get(key);

    if (cached) return JSON.parse(cached) as T;

    // Prevent cache stampede with distributed lock
    const lockKey = `lock:${key}`;
    const lockAcquired = await this.lockClient.set(lockKey, '1', 'EX', 5, 'NX');
    if (!lockAcquired) {
      // Backoff and retry read
      await new Promise(r => setTimeout(r, 100));
      const retry = await this.client.get(key);
      if (retry) return JSON.parse(retry) as T;
    }

    try {
      const value = await fetchFn();
      const ttl = this.applyJitter(opts?.ttl ?? this.defaultTTL);
      await this.client.set(key, JSON.stringify(value), 'EX', ttl);
      return value;
    } finally {
      await this.lockClient.del(lockKey);
    }
  }

  async close(): Promise<void> {
    await this.client.quit();
    await this.lockClient.quit();
  }
}

Step 3: Architecture Rationale

• Mutex Locking: The getOrSet method uses a distributed lock (lock:key) to serialize cache misses. Without this, concurrent requests for the same missing key trigger identical database queries, causing a stampede.
• TTL Jitter: Subtracting 10–20% random jitter from TTLs prevents mass expiration events that overwhelm the primary datastore.
• Versioned Keys: Appending a version string enables atomic invalidation. Bumping the version creates a new key namespace, allowing stale entries to expire naturally without scan operations.
• Separate Lock Client: Isolating lock operations prevents lock contention from blocking standard read/write pipelines.

Pitfall Guide

1. Static TTLs on Volatile Data

Problem: Assigning uniform TTLs regardless of data mutation frequency causes either excessive staleness or unnecessary cache churn. Mitigation: Implement dynamic TTLs based on data classification. Static reference data: 24–72h. User profiles: 5–15m. Transactional state: 0–60s or cache-aside with explicit invalidation.

2. Cache Stampedes

Problem: Multiple concurrent requests miss the cache simultaneously, hammering the backend. Mitigation: Use distributed mutex locks, probabilistic early expiration (refresh at 80% TTL), or stale-while-revalidate patterns. Never allow uncoordinated cache misses to propagate to the database.

3. Key Namespace Collisions

Problem: Unprefixed or monotonically incrementing keys cause cross-service pollution and invalidation failures. Mitigation: Enforce strict key schemas: {env}:{service}:{version}:{entity}:{identifier}. Use Redis KEYS or SCAN only in maintenance windows, never in request paths.

4. Treating Cache as Authoritative

Problem: Applications assume cached data is always fresh and skip validation or fallback logic. Mitigation: Cache is ephemeral. Always implement graceful degradation: if cache fails, route to database with timeout boundaries. Log cache misses as metrics, not errors.

5. Missing Eviction Observability

Problem: Teams monitor latency but ignore eviction rates, hit ratios, and memory fragmentation. Mitigation: Instrument cache_hits, cache_misses, evicted_keys, and memory_used. Alert when hit ratio drops below 60% for read-heavy endpoints or when eviction rate exceeds 5% of total writes.

6. Synchronous Cache Writes Blocking Request Paths

Problem: Writing to cache synchronously adds 10–30ms to every response, negating read optimizations. Mitigation: Decouple cache population from request flow. Use message queues, pub/sub invalidation, or background workers to populate cache after database commits.

7. Ignoring Serialization Overhead

Problem: Repeated JSON serialization/deserialization consumes CPU cycles, especially for nested payloads. Mitigation: Benchmark formats. Switch to MessagePack for high-throughput services. Compress payloads >1KB. Avoid serializing metadata that isn't consumed downstream.

Production Bundle

Action Checklist

• Define cache boundaries: Identify read-heavy, immutable, or eventually consistent data suitable for caching.
• Implement versioned key namespaces: Prevent collision and enable atomic invalidation without scans.
• Add TTL jitter: Randomize expiration by 10–20% to prevent mass eviction storms.
• Deploy distributed mutex for cache misses: Serialize first-touch requests to prevent stampedes.
• Instrument hit/miss ratios and eviction rates: Alert on degradation, not just latency.
• Decouple write-path caching: Use async population or pub/sub to avoid request blocking.
• Test failure modes: Simulate cache unavailability, network partitions, and high miss rates.
• Document invalidation contracts: Explicitly map which events trigger cache deletion or version bumps.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High-read static content (docs, configs)	Cache-Aside + 24h TTL + Versioned Keys	Low mutation, high read volume, simple invalidation	↓ 60–80% DB load
Financial transactions / audit logs	Write-Through or No Cache	Strong consistency required; cache adds latency risk	↑ 10–15% write latency, ↓ data inconsistency risk
Real-time analytics / dashboards	Read-Through + 30s TTL + Stale-While-Revalidate	Balances freshness with throughput; tolerates minor staleness	↓ 40–50% query compute
Multi-region deployment	Redis Cluster + Local Edge Cache + TTL 5m	Reduces cross-region latency; edge cache absorbs regional spikes	↑ 20–30% infra cost, ↓ 60% cross-region traffic

Configuration Template

// redis.config.ts
import { RedisOptions } from 'ioredis';

export const redisOptions: RedisOptions = {
  host: process.env.REDIS_HOST || '127.0.0.1',
  port: parseInt(process.env.REDIS_PORT || '6379', 10),
  password: process.env.REDIS_PASSWORD,
  keyPrefix: `${process.env.NODE_ENV || 'dev'}:backend:`,
  maxRetriesPerRequest: 3,
  retryStrategy: (times) => Math.min(times * 100, 2000),
  connectTimeout: 5000,
  commandTimeout: 3000,
  enableReadyCheck: true,
  enableOfflineQueue: false,
  showFriendlyErrorStack: process.env.NODE_ENV === 'development',
  // Production hardening
  family: 4,
  keepAlive: 30000,
  lazyConnect: true,
};

export const readReplicaOptions: RedisOptions = {
  ...redisOptions,
  host: process.env.REDIS_REPLICA_HOST || redisOptions.host,
  readOnly: true,
};

Quick Start Guide

1. Install dependencies: npm install ioredis @types/ioredis
2. Initialize client: Import redisOptions, instantiate Redis with connection pooling and timeout boundaries.
3. Wrap fetch logic: Replace direct database calls with getOrSet pattern, injecting the fetch function and TTL.
4. Add observability: Export cache_hits and cache_misses counters to your metrics pipeline (Prometheus, Datadog, or OpenTelemetry).
5. Validate failure path: Kill Redis locally, confirm the service falls back to the database within the configured timeout, and logs cache unavailability without crashing.