Digital product scaling

his.readPool = new Pool({ ...readConfig, max: 50 }); this.writePool = new Pool({ ...writeConfig, max: 20 }); }

async findById(id: string): Promise<T | null> { // Route reads to replica pool const client = await this.readPool.connect(); try { const result = await client.query(SELECT * FROM entities WHERE id = $1, [id]); return result.rows[0] || null; } finally { client.release(); } }

async save(entity: T): Promise<T> { // Route writes to primary pool const client = await this.writePool.connect(); try { // Transaction for data integrity await client.query('BEGIN'); const result = await client.query( INSERT INTO entities (id, data) VALUES ($1, $2) ON CONFLICT (id) DO UPDATE SET data = $2 RETURNING *, [(entity as any).id, JSON.stringify(entity)] ); await client.query('COMMIT'); return result.rows[0]; } catch (e) { await client.query('ROLLBACK'); throw e; } finally { client.release(); } } }

### Step 4: Asynchronous Processing and Event-Driven Architecture

Decouple processing from the request lifecycle. Use message queues (e.g., RabbitMQ, Kafka, SQS) to handle background tasks, notifications, and data synchronization. This smooths traffic spikes and improves perceived latency.

**Architecture Decision: Event Sourcing vs. CRUD**
For products requiring audit trails and high scalability of reads, adopt Event Sourcing. Store state changes as immutable events rather than current state. This allows rebuilding state on demand and scaling reads independently. For standard CRUD operations with high write volume, use CQRS (Command Query Responsibility Segregation) to separate write models from read models, optimizing each for its specific workload.

### Step 5: Observability and Auto-Scaling Metrics

Scaling decisions must be data-driven. Implement metrics collection for custom business metrics, not just CPU/Memory. Use metrics like `requests_per_second`, `queue_depth`, and `database_lock_wait_time` to drive auto-scaling policies.

## Pitfall Guide

Scaling implementations often fail due to subtle architectural errors. The following pitfalls are common in production environments and must be avoided.

### 1. Thundering Herd on Cache Invalidation
**Mistake:** When a cache entry expires, thousands of concurrent requests hit the database simultaneously, causing a spike that can crash the database.
**Best Practice:** Implement cache locking or probabilistic early expiration. Use a "stale-while-revalidate" pattern where the cache returns stale data while asynchronously refreshing the value. Ensure cache keys have jittered TTLs to prevent mass expiration.

### 2. Synchronous Chaining of Services
**Mistake:** Building request flows that chain multiple synchronous service calls. Latency multiplies, and the failure probability increases with each hop. A slow downstream service blocks threads and exhausts connection pools.
**Best Practice:** Break synchronous chains using event-driven communication. Use the "Fan-out" pattern for parallel processing where possible. Implement timeouts and circuit breakers on every external call.

### 3. Database Connection Exhaustion
**Mistake:** Creating a new database connection per request without pooling. As concurrency increases, the database runs out of available connections, leading to connection refused errors.
**Best Practice:** Always use connection pooling. Size pools based on the database's `max_connections` and the number of application instances. Monitor pool wait times and utilization. Consider using PgBouncer for connection multiplexing in high-concurrency scenarios.

### 4. Ignoring Idempotency
**Mistake:** Retrying failed requests without idempotency checks causes duplicate transactions, data corruption, and inconsistent state. This is critical in payment processing and inventory management.
**Best Practice:** Implement idempotency keys for all write operations. The client generates a unique key for the request; the server stores processed keys and returns the cached result for duplicate requests. Use database constraints to enforce uniqueness.

### 5. Stateful Services Preventing Horizontal Scaling
**Mistake:** Storing session data or user state in application memory. This prevents auto-scaling because new instances cannot access existing state, and traffic must be sticky-routed, reducing load balancing efficiency.
**Best Practice:** Externalize state to distributed caches (Redis, Memcached) or databases. Design services to be stateless. Use JWTs or session tokens stored in cookies to manage client-side state where appropriate.

### 6. N+1 Query Problems at Scale
**Mistake:** Executing a database query in a loop for each item in a collection. This works for small datasets but causes catastrophic performance degradation as data volume grows.
**Best Practice:** Use batch loading techniques. In GraphQL, use DataLoader to batch and cache requests. In SQL, use `JOIN` clauses or `IN` queries to fetch related data in a single round trip. Profile queries regularly using `EXPLAIN ANALYZE`.

### 7. Lack of Load Testing for Scaling Events
**Mistake:** Assuming the system scales based on theoretical capacity. Real-world traffic patterns, including bursty loads and complex query combinations, often reveal bottlenecks not visible in unit tests.
**Best Practice:** Implement continuous load testing. Use tools like k6 or Gatling to simulate peak traffic scenarios before deployments. Test auto-scaling policies and chaos engineering scenarios to verify resilience.

## Production Bundle

### Action Checklist

- [ ] **Define Scaling Metrics:** Establish clear KPIs for scaling, including max RPS, p99 latency targets, and error rate thresholds.
- [ ] **Implement Circuit Breakers:** Add circuit breakers to all external service calls and database connections to prevent cascading failures.
- [ ] **Enable Read/Write Splitting:** Configure database routing to direct read queries to replicas and write queries to the primary instance.
- [ ] **Add Idempotency Keys:** Ensure all write endpoints accept and validate idempotency keys to support safe retries.
- [ ] **Configure Auto-Scaling Groups:** Set up auto-scaling policies based on custom metrics (e.g., queue depth, CPU utilization) rather than static thresholds.
- [ ] **Review Database Indexes:** Analyze slow query logs and add composite indexes to optimize high-traffic query patterns.
- [ ] **Implement Cache Warming:** Develop strategies to pre-populate caches during low-traffic periods or after deployments to prevent cold-start latency.
- [ ] **Run Load Tests:** Execute load tests simulating 2x expected peak traffic to validate scaling behavior and identify bottlenecks.

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| **High Read / Low Write Ratio** | Aggressive Caching + CDN | Offloads read traffic from origin; reduces database load significantly. | Low: Caching reduces compute and DB costs. |
| **High Write Burst / Variable Load** | Event-Driven Architecture + Message Queues | Decouples ingestion from processing; smooths spikes; ensures durability. | Medium: Queue infrastructure costs, but prevents over-provisioning. |
| **Global User Base / Latency Sensitivity** | Edge Computing + Geo-Distributed DB | Reduces latency by processing data closer to users; improves UX. | High: Multi-region deployment increases infrastructure complexity and cost. |
| **Cost Constraints / Moderate Growth** | Vertical Scaling + Query Optimization | Maximizes existing resources; avoids distributed system overhead. | Low: Immediate cost savings; limited long-term scalability. |
| **Complex Data Relationships / Audit Needs** | Event Sourcing + CQRS | Scales reads/writes independently; provides full audit trail. | High: Increased storage and implementation complexity. |

### Configuration Template

**Kubernetes Horizontal Pod Autoscaler (HPA) with Custom Metrics**

This template configures auto-scaling based on custom metrics (e.g., requests per second) collected by Prometheus.

```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: digital-product-scaler
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: product-service
  minReplicas: 3
  maxReplicas: 50
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "1000" # Scale up if average RPS > 1000 per pod
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
      - type: Percent
        value: 50
        periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60

Quick Start Guide

1. Install k6: Run brew install k6 (macOS) or download from k6.io.
2. Create Load Test Script: Write a load_test.js script simulating user workflows:

   import http from 'k6/http';
   import { check, sleep } from 'k6';
   
   export const options = {
     stages: [
       { duration: '2m', target: 100 },
       { duration: '5m', target: 500 },
       { duration: '2m', target: 0 },
     ],
   };
   
   export default () => {
     const res = http.get('https://your-api.com/products');
     check(res, { 'status is 200': (r) => r.status === 200 });
     sleep(1);
   };
   

3. Run Baseline Test: Execute k6 run load_test.js. Analyze the output for latency spikes, error rates, and throughput limits.
4. Identify Bottleneck: Review application logs, database query times, and CPU/Memory usage during the test. If latency increases linearly with load, check for database locks or connection pool limits.
5. Apply Optimization: Implement a fix (e.g., add cache, optimize query, increase pool size). Re-run the test to validate improvement. Iterate until scaling targets are met.