ffmpeg-pipeline-config.yaml

a decoupled pipeline: ingest, transcode, package, distribute, and playback. Each stage must be stateless, idempotent, and observable.

Step 1: Ingest & Transcode Orchestration

Raw video enters via multipart upload or live RTMP/SRT ingest. A job queue (SQS, RabbitMQ, or Redis) dispatches transcode tasks to stateless workers. Workers use FFmpeg to generate multiple bitrate ladders, enforcing strict GOP alignment and keyframe intervals.

// transcode-worker.ts
import { exec } from 'child_process';
import { promisify } from 'util';
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

const execAsync = promisify(exec);
const s3 = new S3Client({ region: process.env.AWS_REGION });

interface TranscodeJob {
  jobId: string;
  inputPath: string;
  outputBucket: string;
  ladder: { name: string; bitrate: string; resolution: string }[];
}

export async function runTranscodePipeline(job: TranscodeJob): Promise<void> {
  const baseCmd = `-i ${job.inputPath} -c:v libx264 -preset medium -g 48 -keyint_min 48 -sc_threshold 0 -c:a aac -b:a 128k`;
  
  for (const tier of job.ladder) {
    const segmentName = `${job.jobId}/${tier.name}/`;
    const cmd = `${baseCmd} -b:v ${tier.bitrate} -s ${tier.resolution} -f hls -hls_time 2 -hls_list_size 0 -hls_segment_filename ${segmentName}seg%d.ts ${segmentName}playlist.m3u8`;
    
    await execAsync(cmd);
    
    // Upload segments and manifest to S3
    const files = await glob(`${segmentName}*`);
    for (const file of files) {
      const key = file.replace('./', '');
      await s3.send(new PutObjectCommand({
        Bucket: job.outputBucket,
        Key: key,
        Body: require('fs').createReadStream(file),
        ContentType: key.endsWith('.m3u8') ? 'application/x-mpegURL' : 'video/MP2T',
        CacheControl: key.includes('seg') ? 'public, max-age=31536000' : 'no-cache'
      }));
    }
  }
}

Step 2: Packaging & Manifest Generation

HLS manifests must be versioned and updated incrementally. LL-HLS requires <EXT-X-SERVER-CONTROL CAN-BLOCK-RELOAD=YES> and <EXT-X-PRELOAD-HINT> tags. CMAF packaging consolidates audio/video into unified fragments, enabling seamless ABR switching and reducing CDN cache fragmentation.

Architecture decision: Use separate manifests per resolution, linked via a master playlist. This enables CDN edge caching at the resolution level, drastically improving hit ratios. Avoid dynamic manifest generation at the edge unless using specialized media CDNs (e.g., Akamai, Cloudflare Stream).

Step 3: CDN Distribution & Edge Caching

Configure CloudFront or equivalent with:

• Origin shield to reduce origin load
• Cache policies that respect Cache-Control headers
• Query string forwarding disabled for static segments
• TTL: 1 year for segments, 0 for manifests during live

Step 4: Client-Side Playback & Adaptation

Integrate hls.js or video.js with custom ABR logic. The player should probe bandwidth, request segments, and handle manifest updates. For LL-HLS, enable lowLatencyMode: true and configure maxBufferLength appropriately.

// player-config.ts
import Hls from 'hls.js';

export function initializePlayer(videoEl: HTMLVideoElement, manifestUrl: string) {
  if (Hls.isSupported()) {
    const hls = new Hls({
      lowLatencyMode: true,
      maxBufferLength: 10,
      maxMaxBufferLength: 30,
      xhrSetup: (xhr) => {
        xhr.withCredentials = false;
        xhr.timeout = 5000;
      },
      fragLoadingTimeOut: 8000,
      fragLoadingMaxRetry: 3
    });
    hls.loadSource(manifestUrl);
    hls.attachMedia(videoEl);
    hls.on(Hls.Events.ERROR, (event, data) => {
      if (data.fatal) {
        console.error(`[HLS] Fatal: ${data.details}`);
        // Implement fallback or retry logic
      }
    });
  }
}

Step 5: Analytics & Health Monitoring

Track QoE metrics: startup time, rebuffer ratio, bitrate switches, error rates. Ship metrics to Prometheus/Grafana or Datadog. Use client-side beaconing to correlate CDN logs with actual viewer experience.

Pitfall Guide

1. Misaligned GOP/Keyframe Intervals If keyframes don't align across bitrate variants, ABR switching causes visual glitches or playback stalls. FFmpeg must enforce -g 48 -keyint_min 48 -sc_threshold 0 to prevent scene-change keyframes from breaking alignment. Always verify with ffprobe before packaging.

2. Audio/Video Sync Drift Transcoding pipelines that process audio and video separately often drift by 20–200ms. Use -async 1 or -af aresample=async=1 in FFmpeg, and ensure container timestamps are synchronized. Test with mediainfo and player sync metrics.

3. Stale Manifest Caching CDNs caching HLS manifests too aggressively cause clients to request missing segments or miss new live chunks. Set Cache-Control: no-cache for manifests, max-age=31536000 for segments. Implement manifest versioning or use #EXT-X-MEDIA-SEQUENCE tracking.

4. Ignoring DRM & Encryption Premium content without AES-128 or Widevine/FairPlay encryption is trivially scraped. Even if not required initially, design the pipeline to support key rotation and license server integration. Use ffmpeg -hls_key_info_file for HLS encryption.

5. Overcomplicating Real-Time Features Adding WebRTC for non-interactive use cases introduces unnecessary complexity, NAT traversal issues, and higher server costs. Reserve WebRTC for video conferencing or live auctions. Use LL-HLS for live sports, news, or interactive streaming.

6. Neglecting Client-Side Buffering Metrics Server-side logs don't reflect actual viewer experience. Implement client-side telemetry tracking startup time, rebuffer events, and bitrate switches. Correlate with CDN edge logs to identify regional delivery failures.

7. Using Progressive Download for VOD at Scale Progressive MP4 forces clients to download the entire file or guess bitrate. This wastes bandwidth, causes buffering on fluctuating networks, and increases CDN egress. Always use ABR for VOD. Progressive is only acceptable for small, internal, or low-traffic assets.

Production Bundle

Action Checklist

• Define bitrate ladder based on target resolutions and device matrix
• Enforce GOP alignment and keyframe consistency across all variants
• Implement CMAF packaging or LL-HLS for latency-sensitive workflows
• Configure CDN cache policies with strict manifest/segment separation
• Integrate client-side ABR player with custom error handling and telemetry
• Set up QoE monitoring dashboard tracking buffer ratio, startup time, and bitrate switches
• Implement DRM/encryption pipeline if distributing premium or licensed content
• Conduct load testing with synthetic viewers to validate CDN hit ratios and origin scaling

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
VOD library (10k+ assets)	HLS/DASH ABR + CMAF	Maximizes CDN cache hit ratio, reduces egress, supports legacy devices	Lowers CDN cost by 40–60% vs progressive
Live sports/news	LL-HLS + CMAF	Delivers 2–4s latency while preserving ABR and edge caching	+15% infra cost vs traditional HLS, but prevents viewer churn
Interactive/webinar	WebRTC + SFU	Sub-500ms latency, bidirectional audio/video, NAT traversal handled	High server cost ($0.02–0.05/min/consumer), only for <500 concurrent
Internal/training content	Progressive MP4 + CDN	Simpler pipeline, acceptable for controlled networks	Lowest setup cost, but scales poorly beyond 1k viewers

Configuration Template

# ffmpeg-pipeline-config.yaml
pipeline:
  input: s3://raw-bucket/{video_id}/source.mp4
  output: s3://stream-bucket/{video_id}/
  ladder:
    - name: 1080p
      resolution: 1920x1080
      video_bitrate: 4500k
      audio_bitrate: 192k
    - name: 720p
      resolution: 1280x720
      video_bitrate: 2500k
      audio_bitrate: 128k
    - name: 480p
      resolution: 854x480
      video_bitrate: 1200k
      audio_bitrate: 96k
  encoding:
    codec: libx264
    preset: medium
    gop: 48
    keyint_min: 48
    sc_threshold: 0
    hls_time: 2
    hls_list_size: 0
    format: hls
    encryption:
      enabled: true
      key_uri: https://keys.example.com/{video_id}.key
      key_info_file: /tmp/keyinfo.txt
  cdn:
    origin_shield: true
    cache_policy:
      manifest: no-cache
      segment: max-age=31536000, public
    compression: gzip (manifests only)

Quick Start Guide

1. Create Storage & CDN: Provision an S3 bucket for media, enable versioning, and attach a CloudFront distribution. Configure cache behaviors to separate .m3u8 and .ts/.m4s files.
2. Transcode a Sample: Run ffmpeg -i input.mp4 -c:v libx264 -g 48 -keyint_min 48 -sc_threshold 0 -f hls -hls_time 2 -hls_list_size 0 output/playlist.m3u8. Verify GOP alignment with ffprobe.
3. Upload & Distribute: Push segments and manifests to S3. Set correct Cache-Control headers. Point CloudFront origin to the bucket and validate edge caching.
4. Embed Player: Initialize hls.js with the master manifest URL. Test ABR switching by throttling network in DevTools. Verify segment requests and manifest updates.
5. Monitor: Add client-side beaconing for hls.on(Hls.Events.FRAG_LOADED) and Hls.Events.ERROR. Ship metrics to your observability stack. Iterate on ladder and buffer settings based on real QoE data.