Back to KB
Difficulty
Intermediate
Read Time
7 min

Stop Fetching Data Sequentially When It Could Be Parallel

By Codcompass Team··7 min read

Architecting Low-Latency Data Retrieval: Concurrency Patterns for Modern Applications

Current Situation Analysis

Modern applications routinely aggregate data from multiple endpoints before rendering a single view. The architectural expectation is that independent data sources should be retrieved simultaneously. Yet, a persistent pattern of sequential data fetching continues to degrade performance across frontend and server-side codebases. This pattern manifests when developers apply synchronous mental models to asynchronous operations, resulting in request waterfalls that artificially inflate load times.

The problem is frequently overlooked because local development environments mask network latency. When APIs respond in 10-20ms, a chain of five sequential requests completes in under 100ms. The delay is imperceptible, reinforcing the linear coding pattern. However, production environments introduce variable network conditions, cold serverless starts, and geographic latency. Under these conditions, the mathematical penalty of sequential fetching becomes severe.

Consider a standard dashboard that loads a primary resource list, then fetches supplementary metadata for each item. If the list contains 20 items and each metadata request averages 200ms, a linear await loop forces the browser to wait 4,000ms. The data is independent; request #3 does not require the payload of request #2. Yet the execution model queues them. In production, this translates to perceived application sluggishness, higher bounce rates, and unnecessary strain on connection pools. The issue is not a lack of tooling; it is a failure to recognize dependency boundaries and orchestrate promises accordingly.

WOW Moment: Key Findings

The performance delta between sequential and concurrent retrieval is not marginal. It fundamentally alters how an application utilizes network bandwidth and manages user perception. The following comparison isolates the operational characteristics of three common retrieval strategies when handling 20 independent requests averaging 200ms each.

ApproachTotal LatencyError ResilienceNetwork EfficiencyUX Predictability
Sequential Loop (for...await)~4,000msFails silently or crashes on first errorPoor (serialized connections)Low (spinning state persists)
Promise.all~200msFails fast (entire batch rejects)Optimal (parallel connections)High (all-or-nothing)
Promise.allSettled + Concurrency Control~200-350msGraceful degradation (partial success)High (bounded parallelism)High (atomic state commit)

This finding matters because it shifts data fetching from a linear execution problem to a resource orchestration problem. Parallel retrieval does not merely reduce wait times; it enables deterministic loading states, reduces time-to-interactive, and prevents connection pool exhaustion when properly bounded. The architectural win comes from treating independent endpoints as a single logical unit, resolving them concurrently, and committing results atomically to the application state.

Core Solution

Eliminating request waterfalls requires a disciplined approach to promise orchestration, error handling, and state management. The implementation follows four architectural steps.

Step 1: Map Dependencies Explicitly

Before writing fetch logic, audit the data requirements. Identify which endpoints are truly independent. If endpoint B requires an ID, token, or calculated value from endpoint A, the dependency is real and sequential awaiting is correct. If both endpoints can be called with static or pre-existing parameters, they belong in a concurrent batch.

Step 2: Orchestrate with Promise.allSettled

Production systems rarely tolerate all-or-nothing failures. Promise.all rejects immediately if any promise rejects, which can wipe out successfully fetched data due to a single flaky endpoint. Promise.allSettled waits for every promise to resolve or reject, returning an array of status objects. This enables graceful degradation.

interface FetchResult<T> {
  status: 'fulfilled' | 'rejected';
  value?: T;
  reason?: unknown;
}

async function batchFetch<T>(
  endpoints: string[],
  transformer: (response: Response) => Promise<T>
): Promise<FetchResult<T>[]> {
  const promises = endpoints.map(url =>
    fetch(url)
      .then(res => {
        if (!res.ok) throw new Error(`HTTP ${res.status} for ${url}`);
        return transformer(res);
      })
  );

  return Promise.allSettled(promises);
}

Step 3: Normalize and Filter Results

The settled array contains mixed outcomes. Production code must normalize these into a usable structure, logging failures for observability while preserving successful payloads.

function normalizeBatchResults<T>(
  results: FetchResult<T>[],
  fallbackValue: T
): T[] {
  return results.map((result, index) => {
    if (result.status === 'fulfilled') {
      return result.value as T;
    }
    console.warn(`Batch fetch failed at index ${index}:`, result.reason);
    return fallbackValue;
  });
}

Step 4: Commit State Atomically

Fragmented state updates trigger unnecessary re-renders and can expose users to partially loaded UI states. Resolve the entire batch first, then update the application state in a single transaction.

import { useState, useCallback } from 'react';

interface ProductMetrics {
  sk

u: string; inventoryCount: number; lastUpdated: string; }

export function useProductMetrics() { const [metrics, setMetrics] = useState<ProductMetrics[]>([]); const [isLoading, setIsLoading] = useState(false);

const loadMetrics = useCallback(async (skus: string[]) => { setIsLoading(true); try { const endpoints = skus.map(sku => /api/inventory/${sku}); const rawResults = await batchFetch<ProductMetrics>(endpoints, res => res.json()); const normalized = normalizeBatchResults(rawResults, { sku: 'unknown', inventoryCount: 0, lastUpdated: new Date().toISOString() }); setMetrics(normalized); } catch (error) { console.error('Critical batch failure:', error); } finally { setIsLoading(false); } }, []);

return { metrics, isLoading, loadMetrics }; }


### Architecture Rationale
- **Why `Promise.allSettled` over `Promise.all`?** Production APIs experience transient failures. Rejecting an entire batch because one inventory endpoint timed out degrades user experience unnecessarily. Settled promises allow the UI to render available data while marking missing items as stale or unavailable.
- **Why atomic state commits?** React's reconciliation algorithm batches updates within event handlers, but async boundaries break this guarantee. Resolving all data before calling `setState` prevents intermediate renders that display mismatched or incomplete data.
- **Why explicit error normalization?** Silently swallowing errors hides infrastructure issues. Logging at the batch level provides traceability without crashing the rendering cycle.

## Pitfall Guide

### 1. Unbounded Concurrency
**Explanation:** Firing hundreds of parallel requests simultaneously exhausts browser connection limits (typically 6 per origin) and triggers server-side rate limiting or connection resets.
**Fix:** Implement a concurrency limiter that processes promises in chunks. Use a sliding window or a promise queue to cap active requests at 10-20 depending on the target environment.

### 2. The Fail-Fast Trap
**Explanation:** Relying on `Promise.all` in user-facing flows causes complete data loss when a single non-critical endpoint fails.
**Fix:** Default to `Promise.allSettled` for independent data. Reserve `Promise.all` for strict transactional workflows where partial data is invalid.

### 3. State Fragmentation & Render Thrashing
**Explanation:** Updating state incrementally as each promise resolves causes multiple re-renders and can display inconsistent UI states (e.g., a list where some items show data and others show placeholders indefinitely).
**Fix:** Resolve the entire batch before calling state setters. Use a single loading flag and a single data payload.

### 4. Hidden Dependency Chains
**Explanation:** Developers sometimes parallelize requests that actually share implicit dependencies, such as authentication tokens, session cookies, or calculated query parameters.
**Fix:** Audit parameter sources. If request B requires a value derived from request A's response, maintain sequential awaiting for that specific pair. Parallelize only truly independent branches.

### 5. Rate Limit Blindness
**Explanation:** Third-party APIs and internal gateways enforce strict request quotas. Unchecked parallelism triggers 429 responses, forcing retries that compound latency.
**Fix:** Respect `Retry-After` headers. Implement exponential backoff with jitter. For high-volume batches, use bulk endpoints or GraphQL batching when available.

### 6. The `.forEach` Async Fallacy
**Explanation:** Using `.forEach()` with async callbacks does not wait for promises to complete. The function returns immediately, leaving fetches dangling and untracked.
**Fix:** Replace `.forEach()` with `.map()` to capture promises, then aggregate with `Promise.all` or `Promise.allSettled`. Never use `.forEach()` for asynchronous operations.

### 7. Ignoring Cache Invalidation
**Explanation:** Parallel fetching bypasses natural request deduplication. If multiple components request the same endpoint simultaneously, the browser may issue duplicate network calls instead of sharing a single response.
**Fix:** Implement a lightweight request deduplication layer or use a data-fetching library that caches in-flight promises. Ensure stale-while-revalidate strategies are configured for independent endpoints.

## Production Bundle

### Action Checklist
- [ ] Audit component data requirements and map explicit dependency graphs
- [ ] Replace `for...await` loops with `.map()` + `Promise.allSettled` for independent endpoints
- [ ] Implement a concurrency limiter for batches exceeding 15 requests
- [ ] Normalize settled results into a consistent data structure before state updates
- [ ] Commit all resolved data to state in a single transaction
- [ ] Add structured logging for rejected promises to enable observability
- [ ] Verify browser connection limits and server rate quotas before scaling batch sizes
- [ ] Implement request deduplication to prevent duplicate network calls for identical URLs

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| < 10 independent endpoints | `Promise.allSettled` | Simple orchestration, full resilience | Negligible |
| 10-50 independent endpoints | Chunked `Promise.allSettled` (batch size 15) | Prevents connection exhaustion and rate limits | Low (slight latency increase for batching) |
| Strict transactional data | `Promise.all` | Fails fast to maintain data consistency | Medium (higher failure rate on flaky networks) |
| Third-party rate-limited API | Sequential with exponential backoff | Complies with quota limits, avoids 429s | High (latency scales linearly) |
| Real-time dashboard updates | WebSocket or Server-Sent Events | Eliminates polling overhead entirely | High (infrastructure complexity) |

### Configuration Template

```typescript
// concurrent-fetcher.ts
export class ConcurrentFetcher {
  private concurrencyLimit: number;
  private activeRequests: number;
  private queue: Array<() => Promise<void>>;

  constructor(limit: number = 10) {
    this.concurrencyLimit = limit;
    this.activeRequests = 0;
    this.queue = [];
  }

  async execute<T>(task: () => Promise<T>): Promise<T> {
    return new Promise<T>((resolve, reject) => {
      const run = async () => {
        this.activeRequests++;
        try {
          const result = await task();
          resolve(result);
        } catch (error) {
          reject(error);
        } finally {
          this.activeRequests--;
          this.processQueue();
        }
      };

      if (this.activeRequests < this.concurrencyLimit) {
        run();
      } else {
        this.queue.push(run);
      }
    });
  }

  private processQueue(): void {
    while (this.queue.length > 0 && this.activeRequests < this.concurrencyLimit) {
      const next = this.queue.shift();
      if (next) next();
    }
  }

  async batch<T>(tasks: Array<() => Promise<T>>): Promise<PromiseSettledResult<T>[]> {
    const promises = tasks.map(task => this.execute(task));
    return Promise.allSettled(promises);
  }
}

// Usage example
const fetcher = new ConcurrentFetcher(12);
const endpoints = ['/api/a', '/api/b', '/api/c'];
const results = await fetcher.batch(
  endpoints.map(url => () => fetch(url).then(r => r.json()))
);

Quick Start Guide

  1. Identify Independent Endpoints: List all API calls required for your view. Mark which ones share parameters or responses. Isolate the independent group.
  2. Replace Linear Awaits: Convert for...await or chained .then() calls into a .map() that returns promises. Wrap the array in Promise.allSettled().
  3. Normalize Results: Iterate through the settled array. Extract value from fulfilled promises, log reason from rejected ones, and map to a consistent interface.
  4. Update State Once: Pass the normalized array to your state setter. Remove intermediate loading flags per-item. Use a single isLoading boolean for the entire batch.
  5. Validate in Production Conditions: Test on throttled networks (3G/4G simulation) and with artificial latency. Verify that total load time matches the slowest single request, not the sum of all requests.