Saga pattern for distributed transactions

new Date() };

await this.stateStore.save(execution);

try {
  for (let i = 0; i < this.steps.length; i++) {
    execution.currentStepIndex = i;
    await this.stateStore.save(execution);

    await this.steps[i].execute(payload);
    execution.completedSteps.push(this.steps[i].name);
    await this.stateStore.save(execution);
  }

  execution.state = 'COMPLETED';
} catch (error) {
  execution.state = 'COMPENSATING';
  execution.error = error as Error;
  await this.stateStore.save(execution);

  await this.compensate(execution);

  execution.state = 'FAILED';
  execution.updatedAt = new Date();
  await this.stateStore.save(execution);
  throw error;
}

execution.updatedAt = new Date();
await this.stateStore.save(execution);
return execution;

}

private async compensate(execution: SagaExecution<T>): Promise<void> { for (let i = execution.completedSteps.length - 1; i >= 0; i--) { const stepName = execution.completedSteps[i]; const step = this.steps.find(s => s.name === stepName)!; try { await step.compensate(execution.payload); } catch (compError) { // Log and alert. Do not block other compensations. console.error(Compensation failed for ${stepName}, compError); } } } }

### Step 4: Integrate Idempotency and Outbox Pattern
Sagas operate in distributed environments where network retries are inevitable. Every step must accept an idempotency key derived from the saga execution ID and step index. Steps should publish domain events via an outbox table to guarantee at-least-once delivery without blocking the transaction.

```typescript
export async function createIdempotencyKey(sagaId: string, stepIndex: number): Promise<string> {
  return `${sagaId}:step:${stepIndex}`;
}

Architecture Decisions and Rationale

• Orchestration over Choreography: Centralized state enables deterministic recovery, simplifies testing, and provides a single point for metrics and tracing. Choreography scales horizontally but requires distributed tracing and complex compensation ordering.
• Persistent State Store: In-memory state fails on process restarts. Use Redis, PostgreSQL, or DynamoDB with conditional writes to prevent duplicate executions.
• Forward Recovery Semantics: Compensations apply new state rather than reversing it. This aligns with event-sourcing principles and avoids distributed rollback locks.
• Isolated Compensation Execution: Compensations run independently. A single compensation failure must not block others, and retries must be bounded with exponential backoff.
• Timeout Enforcement: Each step enforces a deadline. Timeouts trigger compensations immediately, preventing resource leakage.

Pitfall Guide

1. Treating Compensation as Rollback Compensations do not reverse database state; they apply corrective state. Assuming rollback semantics leads to inconsistent data when external systems (payment gateways, shipping APIs) cannot be rolled back. Always design compensations as forward state transitions.

2. Missing Idempotency Guarantees Network retries, orchestrator restarts, and message broker redeliveries cause duplicate step invocations. Without idempotency keys, you get double charges, duplicated inventory deductions, or orphaned records. Enforce idempotency at the database or API gateway level.

3. Chaining Compensations Without Isolation Compensations should execute independently. If compensation A fails and blocks compensation B, you create a cascading failure that leaves the system in an unrecoverable state. Run compensations in parallel or sequentially with isolated error handling.

4. Ignoring Partial Success States A saga may complete some steps, fail later, and successfully compensate earlier steps, yet leave external resources provisioned (e.g., a cloud VM created but not billed). Map every step to its resource lifecycle and verify compensation fully releases external allocations.

5. Overcomplicating the State Machine Sagas are linear or DAG-based by design. Introducing cycles, conditional branching within the orchestrator, or dynamic step generation based on runtime data breaks deterministic recovery. Keep the execution graph static; handle business logic inside step implementations.

6. Confusing Timeouts with Failures Network latency spikes cause timeouts that are not actual failures. Implement jittered retries for transient errors before triggering compensation. Use circuit breakers on external calls to distinguish between slow responses and hard failures.

7. Skipping Dead-Letter Queues for Compensation Failures When a compensation fails repeatedly, the saga enters a terminal failed state. Without a dead-letter queue or manual reconciliation workflow, data drift accumulates. Route failed compensations to a dedicated queue with alerting and automated reconciliation scripts.

Best Practices from Production:

• Persist saga state after every step completion and compensation attempt.
• Attach a traceId to all step invocations and compensations for distributed tracing.
• Use outbox pattern for event publishing to guarantee consistency between local DB and message broker.
• Implement circuit breakers and bulkheads on external service calls within steps.
• Run chaos engineering tests that inject network partitions and partial failures during saga execution.

Production Bundle

Action Checklist

• Define explicit compensations for every business step; verify they are idempotent and forward-recovering
• Persist saga execution state after each step completion and compensation attempt
• Implement idempotency keys derived from saga ID and step index at the storage layer
• Route compensation failures to a dead-letter queue with alerting and reconciliation workflows
• Attach distributed trace IDs to all step executions and compensations for observability
• Enforce step-level timeouts with jittered retries before triggering compensation
• Validate saga behavior under network partitions, process restarts, and broker redeliveries

Decision Matrix

Configuration Template

export interface SagaConfig {
  stateStore: {
    type: 'redis' | 'postgres' | 'dynamodb';
    connectionString: string;
    ttlMinutes: number;
  };
  execution: {
    maxRetries: number;
    retryBaseDelayMs: number;
    stepTimeoutMs: number;
    compensationTimeoutMs: number;
  };
  observability: {
    enableTracing: boolean;
    traceServiceName: string;
    metricsPrefix: string;
  };
  idempotency: {
    keyPrefix: string;
    ttlHours: number;
  };
}

export const defaultSagaConfig: SagaConfig = {
  stateStore: {
    type: 'postgres',
    connectionString: process.env.SAGA_DB_URL || '',
    ttlMinutes: 1440 // 24 hours
  },
  execution: {
    maxRetries: 3,
    retryBaseDelayMs: 500,
    stepTimeoutMs: 10000,
    compensationTimeoutMs: 15000
  },
  observability: {
    enableTracing: true,
    traceServiceName: 'saga-orchestrator',
    metricsPrefix: 'saga'
  },
  idempotency: {
    keyPrefix: 'saga:idem',
    ttlHours: 48
  }
};

Quick Start Guide

1. Define Steps: Create TypeScript classes implementing SagaStep for each business operation (e.g., CreateOrderStep, ReserveInventoryStep, ProcessPaymentStep). Implement execute() and compensate() with idempotency checks.
2. Wire Orchestrator: Instantiate SagaOrchestrator with your steps and a persistent state store. Pass the configuration template with environment-specific connection strings and timeouts.
3. Execute Saga: Call orchestrator.execute(sagaId, payload) from your API handler. Handle the returned SagaExecution state to respond to clients. Route failures to compensation queues.
4. Add Observability: Instrument step executions with traceId headers. Export metrics for saga.steps.completed, saga.steps.compensated, and saga.failures. Set alerts on compensation failure rates exceeding 2%.