Handling Duplicate Shopify Webhook Events (And Why You Must)

import { extractResourceFingerprint } from '../utils';

const router = Router();

router.post('/incoming/:topic', async (req: Request, res: Response) => { try { // 1. Verify authenticity immediately if (!verifyShopifySignature(req)) { return res.status(401).send('Unauthorized'); }

const { topic } = req.params;
const payload = req.body;

// 2. Extract stable identifier from payload
// Never use X-Shopify-Webhook-Id here
const fingerprint = extractResourceFingerprint(topic, payload);

// 3. Acknowledge immediately to stop Shopify retries
res.status(200).json({ status: 'accepted', fingerprint });

// 4. Offload processing to queue
// The queue handles retries, backoff, and worker scaling
await EventQueue.publish('shopify.events', {
  fingerprint,
  payload,
  receivedAt: new Date().toISOString()
});

} catch (error) { // Even on error, return 2xx if possible to prevent infinite retries // unless the error is structural (e.g., invalid payload) console.error('Webhook ingestion failed:', error); res.status(200).send('Error logged'); } });

export default router;

#### 2. Stable Fingerprint Construction

The deduplication key must be derived from data that does not change during retries. The combination of the webhook topic and the resource ID provides a globally unique, stable fingerprint.

```typescript
// utils/fingerprint.ts
export function extractResourceFingerprint(topic: string, payload: any): string {
  // Shopify payloads nest IDs differently based on topic
  // e.g., orders/paid has payload.id, inventory_levels/update has payload.id
  const resourceId = payload.id || payload.order_id || payload.product_id;
  
  if (!resourceId) {
    throw new Error('Payload missing stable resource identifier');
  }

  // Format: shopify:{topic}:{resource_id}
  // Example: shopify:orders/paid:gid://shopify/Order/12345
  return `shopify:${topic}:${resourceId}`;
}

3. Atomic Deduplication with Database Safety Net

Redis provides speed, but it is volatile. The database must be the source of truth. Use a unique constraint with an atomic insert to handle race conditions where multiple workers might process the same message simultaneously.

-- migrations/001_create_webhook_receipts.sql
CREATE TABLE webhook_receipts (
  fingerprint VARCHAR(255) PRIMARY KEY,
  topic VARCHAR(100) NOT NULL,
  processed_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
  payload_hash VARCHAR(64) NOT NULL
);

CREATE INDEX idx_webhook_receipts_topic ON webhook_receipts(topic);

// services/deduplication.service.ts
import { db } from '../database';
import { cache } from '../cache';

export class DeduplicationService {
  // TTL must exceed Shopify's maximum retry window (48 hours)
  // 48 hours = 172,800 seconds
  private static readonly DEDUP_TTL_SECONDS = 172800;

  async processWithDedup(fingerprint: string, payload: any, handler: () => Promise<void>): Promise<boolean> {
    // 1. Fast path: Check Redis cache
    const cached = await cache.get(fingerprint);
    if (cached === 'processed') {
      return false; // Duplicate detected
    }

    // 2. Slow path: Atomic database insert
    // ON CONFLICT ensures only one worker succeeds even under concurrency
    const result = await db.query(
      `INSERT INTO webhook_receipts (fingerprint, topic, payload_hash)
       VALUES ($1, $2, $3)
       ON CONFLICT (fingerprint) DO NOTHING
       RETURNING fingerprint`,
      [fingerprint, fingerprint.split(':')[1], this.hashPayload(payload)]
    );

    if (result.rows.length === 0) {
      // Conflict occurred: Another worker processed this
      // Sync cache to prevent future DB hits
      await cache.set(fingerprint, 'processed', { ttl: DeduplicationService.DEDUP_TTL_SECONDS });
      return false;
    }

    // 3. Execute business logic
    await handler();

    // 4. Sync cache after successful processing
    await cache.set(fingerprint, 'processed', { ttl: DeduplicationService.DEDUP_TTL_SECONDS });
    
    return true;
  }

  private hashPayload(payload: any): string {
    // Implementation of SHA-256 hash for payload verification
    return 'hash_placeholder';
  }
}

4. Idempotent Business Logic

Deduplication catches duplicates at the infrastructure level, but your business logic must also be idempotent. This provides defense-in-depth. If deduplication fails (e.g., database outage), the logic should still produce correct results.

Critical Rule: Never use relative mutations for state changes. Always use absolute values derived from the payload.

// handlers/inventory.handler.ts
import { InventoryRepository } from '../repositories';

export async function handleInventoryUpdate(payload: any) {
  const productId = payload.product_id;
  const locationId = payload.location_id;
  
  // ❌ DANGEROUS: Relative mutation breaks on duplicates
  // await InventoryRepository.decrementStock(productId, 5);

  // ✅ SAFE: Absolute update is idempotent
  // The payload contains the authoritative state
  const newStockLevel = payload.available;
  
  await InventoryRepository.setStockLevel({
    productId,
    locationId,
    quantity: newStockLevel
  });
}

Pitfall Guide

Pitfall Name	Explanation	Fix
The Ephemeral Header Trap	Using X-Shopify-Webhook-Id as the dedup key. This header changes on every retry attempt, causing every retry to be processed as a new event.	Always derive the dedup key from payload.id combined with the topic.
The 5-Second Cliff	Performing database writes or external API calls inside the HTTP handler. If processing exceeds 5 seconds, Shopify retries, causing duplicates.	Return 200 OK immediately. Offload all work to a message queue.
Relative State Mutation	Using UPDATE inventory SET qty = qty - 1. If executed twice, inventory drops by 2 instead of 1.	Use UPDATE inventory SET qty = $1 with the absolute value from the webhook payload.
Redis-Only Deduplication	Relying solely on Redis for deduplication. Redis is volatile; a restart or eviction can cause duplicates to slip through.	Use the database with a unique constraint as the source of truth. Redis is a cache for performance.
TTL Mismatch	Setting Redis TTL shorter than Shopify's retry window (e.g., 24 hours). A retry at hour 25 finds no cache entry and re-processes.	Set TTL to at least 48 hours (172,800 seconds) to cover the full retry window.
Ignoring Retry Headers	Failing to log X-Shopify-Retry headers. This header indicates a retry, which is valuable for debugging latency issues.	Log the retry count and timestamp. Alert if retry rates spike, indicating performance degradation.
Race Conditions on Dedup	Checking Redis, then inserting into DB without atomicity. Two concurrent requests can both pass the Redis check and insert duplicates.	Use INSERT ... ON CONFLICT DO NOTHING in the database. The DB constraint is the lock.

Production Bundle

Action Checklist

• Verify Fingerprint Stability: Audit all webhook handlers to ensure dedup keys use payload.id, not headers.
• Implement Async Ack: Ensure all endpoints return 2xx within 2 seconds by offloading work to a queue.
• Add Database Constraint: Create webhook_receipts table with a unique index on the fingerprint column.
• Audit Mutations: Review all inventory, order, and customer updates. Replace relative increments/decrements with absolute sets.
• Align TTLs: Configure cache TTL to ≥ 48 hours to match Shopify's retry window.
• Monitor Retry Rates: Instrument logging to track X-Shopify-Retry headers. Alert on abnormal retry spikes.
• Load Test Duplicates: Simulate concurrent duplicate requests to verify atomic deduplication holds under load.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High Volume Merchant	Redis Cache + DB Atomic Insert	Redis absorbs read load; DB ensures correctness. Essential for >100 events/sec.	Medium (Redis infrastructure)
Low Volume / MVP	DB Atomic Insert Only	Simplicity reduces operational overhead. DB can handle moderate concurrency with unique constraints.	Low
Financial Criticality	DB-First Dedup	Eliminates cache dependency for deduplication. Slower but guarantees no double-processing.	Low (Higher DB load)
Complex Payloads	Payload Hash Verification	Store hash of payload in DB. Detects if Shopify sends the same ID with different data (rare but possible).	Medium (Storage/Compute)

Configuration Template

Database Migration:

-- Ensure this runs before deploying webhook handlers
BEGIN;

CREATE TABLE IF NOT EXISTS webhook_receipts (
  fingerprint VARCHAR(255) PRIMARY KEY,
  topic VARCHAR(100) NOT NULL,
  payload_hash VARCHAR(64) NOT NULL,
  processed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  retry_count INT DEFAULT 0
);

-- Optimize for topic-based queries and cleanup
CREATE INDEX idx_webhook_receipts_topic_processed 
  ON webhook_receipts(topic, processed_at);

-- Optional: Partition by month for high-volume tables
-- CREATE TABLE webhook_receipts_y2024m01 PARTITION OF webhook_receipts
--   FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');

COMMIT;

Environment Configuration:

# .env.example
# Shopify Webhook Configuration
SHOPIFY_WEBHOOK_SECRET=whsec_...
SHOPIFY_RETRY_TTL_SECONDS=172800
SHOPIFY_ACK_TIMEOUT_MS=2000

# Queue Configuration
QUEUE_PROVIDER=bullmq
QUEUE_REDIS_URL=redis://localhost:6379

Quick Start Guide

1. Create Receipt Table: Run the migration to add webhook_receipts with the unique fingerprint constraint.
2. Wrap Handler: Refactor your webhook endpoint to verify the signature, extract the fingerprint, return 200 OK, and publish to your queue.
3. Implement Worker: Create a queue consumer that calls DeduplicationService.processWithDedup. Pass your business logic as the handler callback.
4. Test Duplicates: Use curl to send the same payload twice rapidly. Verify the database contains only one receipt and business logic executed once.
5. Deploy & Monitor: Deploy to staging. Trigger a webhook and verify the response time is <2s. Check logs for X-Shopify-Retry headers to ensure no retries are occurring.