CodcompassCodcompass
← Back to Blog
DevOps2026-05-14·70 min read

AVIF encoding speed — the numbers nobody talks about

By Serhii Kalyna

The Hidden Compute Tax of Next-Gen Image Formats: Engineering Pipelines for Production

Current Situation Analysis

The modern web infrastructure conversation around image formats has been dominated by a single metric: file size. Engineering teams routinely migrate from JPEG to WebP or AVIF to reduce bandwidth costs and improve Core Web Vitals. Benchmarks consistently show that AVIF delivers roughly 50% size reduction compared to JPEG, while WebP achieves approximately 31%. When normalized to perceptual quality (DSSIM), AVIF typically produces files half the size of WebP. These numbers are accurate, but they represent only the delivery side of the equation.

The operational blind spot lies in the encoding phase. Most format comparisons test single-image throughput on isolated workstations. Production environments, however, handle concurrent upload bursts, variable image dimensions, and strict latency SLAs. When you shift from benchmarking to real-world workloads, the compute and memory overhead of next-gen encoders becomes the primary constraint.

The default AVIF encoder (libaom) is computationally intensive. Encoding a standard 1080p image typically requires 1 to 4 seconds, consumes up to 400% CPU across four cores, and spikes memory usage to approximately 2.5GB per job. In contrast, WebP encoding completes in roughly 90 milliseconds, uses about 20% CPU, and peaks near 200MB RAM. At comparable quality settings, AVIF encoding can be up to 47 times slower than WebP. Pushing AVIF to maximum quality settings can extend encoding time to 48 seconds per image.

This discrepancy is rarely discussed because compression benchmarks are publicly visible, while encoding costs are buried in infrastructure metrics. Teams that adopt AVIF for all workloads without architectural adjustments quickly encounter container OOM kills, queue backpressure, and degraded user experience during peak traffic. The problem is misunderstood as a "format choice" issue when it is fundamentally a capacity planning and routing problem.

WOW Moment: Key Findings

The following comparison isolates the operational trade-offs between the most common image encoding strategies. Data reflects 1080p source images processed on identical hardware using libvips/Sharp bindings.

Approach Encoding Latency Peak Memory CPU Utilization Size Reduction vs JPEG
WebP (libwebp) ~90ms ~200MB ~20% ~31%
AVIF (libaom, default) 1–4s ~2.5GB ~400% ~50%
AVIF (SVT-AV1) ~0.5–2s ~1.8GB ~300% ~48–50%
AVIF (libaom, max effort) up to 48s ~2.5GB+ ~400% ~52–54%

Why this matters: The table reveals that AVIF's compression advantage comes with a steep compute tax. WebP remains the optimal choice for latency-sensitive, compute-constrained environments. AVIF excels when encoding time is decoupled from user interaction. SVT-AV1 offers a meaningful middle ground, reducing encoding time by roughly 50% compared to libaom while maintaining near-identical compression ratios. Understanding these trade-offs enables workload-aware routing instead of blanket format adoption.

Core Solution

Building a production-ready image pipeline requires separating static asset generation from dynamic user uploads, selecting encoders based on latency budgets, and enforcing concurrency controls. The following architecture demonstrates a dual-path routing system implemented in TypeScript using sharp.

Architecture Decisions

  1. Static vs Dynamic Routing: Pre-generate AVIF during build or CI pipelines where latency is irrelevant. Encode WebP on-the-fly for user uploads to preserve response times.
  2. Encoder Selection: Use libaom for maximum compression when time permits. Switch to SVT-AV1 for faster throughput with minimal quality loss. Reserve libwebp for real-time paths.
  3. Quality over Effort: The quality parameter directly controls bitrate and file size. The effort parameter only dictates how aggressively the encoder searches for compression optimizations. Higher effort does not guarantee smaller files and can occasionally increase output size due to encoder heuristics.
  4. Concurrency Limiting: AVIF jobs must be isolated in dedicated worker pools with strict memory caps. WebP jobs can share general-purpose compute nodes.

Implementation

The following TypeScript module demonstrates a router that directs images to the appropriate encoder based on workload type, enforces concurrency limits, and logs operational metrics.

import sharp from 'sharp';
import { EventEmitter } from 'events';

interface EncodeConfig {
  format: 'webp' | 'avif';
  quality: number;
  effort: number;
  encoder?: 'libwebp' | 'libaom' | 'svt-av1';
}

interface PipelineMetrics {
  jobId: string;
  format: string;
  latencyMs: number;
  memoryPeakMB: number;
  outputSizeBytes: number;
}

class ImagePipelineRouter extends EventEmitter {
  private activeJobs: number = 0;
  private maxConcurrentAVIF: number;
  private maxConcurrentWebP: number;

  constructor(avifLimit: number = 2, webpLimit: number = 10) {
    super();
    this.maxConcurrentAVIF = avifLimit;
    this.maxConcurrentWebP = webpLimit;
  }

  async routeAndEncode(
    sourceBuffer: Buffer,
    workloadType: 'static' | 'dynamic',
    jobId: string
  ): Promise<Buffer> {
    const isAVIFPath = workloadType === 'static';
    const config: EncodeConfig = isAVIFPath
      ? { format: 'avif', quality: 72, effort: 5, encoder: 'svt-av1' }
      : { format: 'webp', quality: 75, effort: 4, encoder: 'libwebp' };

    if (isAVIFPath && this.activeJobs >= this.maxConcurrentAVIF) {
      throw new Error('AVIF concurrency limit reached. Queue the job.');
    }

    this.activeJobs++;
    const startTime = performance.now();
    const memBefore = process.memoryUsage().heapUsed;

    try {
      const result = await this.executeEncode(sourceBuffer, config);
      const latency = performance.now() - startTime;
      const memPeak = (process.memoryUsage().heapUsed - memBefore) / (1024 * 1024);

      this.emit('metrics', {
        jobId,
        format: config.format,
        latencyMs: Math.round(latency),
        memoryPeakMB: Math.round(memPeak),
        outputSizeBytes: result.length
      } as PipelineMetrics);

      return result;
    } finally {
      this.activeJobs--;
    }
  }

  private async executeEncode(source: Buffer, config: EncodeConfig): Promise<Buffer> {
    const transformer = sharp(source);

    if (config.format === 'avif') {
      return transformer.avif({
        quality: config.quality,
        effort: config.effort,
        chromaSubsampling: '4:4:4'
      }).toBuffer();
    }

    return transformer.webp({
      quality: config.quality,
      effort: config.effort
    }).toBuffer();
  }
}

export default ImagePipelineRouter;

Why this structure works:

  • The router explicitly separates static and dynamic workloads, preventing real-time requests from blocking on heavy AVIF encoding.
  • Concurrency limits are enforced at the application layer, complementing container-level memory restrictions.
  • Metrics emission enables monitoring of p95 latency, memory RSS, and queue depth without coupling to external APM tools.
  • Chroma subsampling is explicitly set to 4:4:4 for AVIF to preserve gradient fidelity, a common oversight that causes banding in photography.

Pitfall Guide

1. The Effort Illusion

Explanation: Developers frequently set effort: 9 expecting proportional file size reductions. In reality, effort controls encoder search depth. At fixed quality, higher effort yields marginal or unpredictable size changes and can occasionally increase output size due to rate-distortion optimization quirks. Fix: Cap effort at 4–6 for production. Treat quality as the primary size controller. Validate output sizes across effort levels before committing to high values.

2. RAM Spikes Under Concurrency

Explanation: AVIF encoding with libaom allocates large frame buffers and reference tables. Running multiple jobs simultaneously without limits causes OOM kills, especially in containerized environments with default memory quotas. Fix: Implement application-level concurrency caps. Deploy AVIF workers in isolated pods with resources.limits.memory set to 3GB. Use a message queue (Redis, RabbitMQ, SQS) to buffer excess jobs.

3. Encoder Lock-in

Explanation: Sticking with libaom because it's the default ignores faster alternatives. SVT-AV1 achieves comparable compression with roughly half the encoding time and lower memory footprint. Fix: Benchmark SVT-AV1 in your environment. Update Sharp/libvips to versions with SVT-AV1 support. Configure environment variables to switch encoders without code changes.

4. Latency Mismatch

Explanation: Using AVIF for user-uploaded profile pictures or real-time transforms introduces 1–4 second delays per image. Users perceive this as application slowness, regardless of bandwidth savings. Fix: Route dynamic uploads to WebP. Pre-generate AVIF variants during build, CI, or background worker cycles. Serve both via <picture> or CDN content negotiation.

5. Fallback Chain Breakage

Explanation: Misconfigured <picture> elements or missing MIME types cause browsers to skip AVIF/WebP and fall back to JPEG, negating compression gains. Fix: Always include a JPEG fallback. Verify CDN Content-Type headers match the encoded format. Test with browser dev tools network panels to confirm format selection.

6. Quality vs Effort Confusion

Explanation: Treating effort and quality as interchangeable leads to inconsistent output sizes. Quality directly maps to quantization tables; effort only affects encoder runtime. Fix: Document encoding presets explicitly. Example: preset: { quality: 72, effort: 5 } for AVIF, preset: { quality: 75, effort: 4 } for WebP. Never adjust effort to control file size.

7. Queue Backpressure Neglect

Explanation: Heavy AVIF jobs accumulate in memory queues when consumer throughput drops. Without backpressure handling, the application crashes or drops jobs silently. Fix: Use bounded queues with explicit rejection policies. Implement exponential backoff for retries. Monitor queue depth and scale worker replicas automatically based on backlog size.

Production Bundle

Action Checklist

  • Route static assets to AVIF pre-generation pipelines; route dynamic uploads to WebP
  • Set AVIF concurrency limits to 2–4 workers per node; WebP can handle 8–12
  • Configure quality: 72 and effort: 5 for AVIF; quality: 75 and effort: 4 for WebP
  • Deploy SVT-AV1 encoder where available; fall back to libaom only if compatibility requires it
  • Implement <picture> fallback chain: AVIF → WebP → JPEG
  • Set container memory limits to 3GB for AVIF workers; monitor RSS with Prometheus/cAdvisor
  • Add queue depth and p95 encoding latency to monitoring dashboards
  • Validate output sizes across effort levels before production deployment

Decision Matrix

Scenario Recommended Approach Why Cost Impact
Real-time user uploads WebP dynamic encoding Sub-100ms latency, low memory footprint Minimal compute cost, preserves UX
Build-time static assets AVIF pre-generation Maximum compression, zero user latency Higher CI/build time, lower bandwidth
Low-memory servers (<2GB) WebP only AVIF spikes exceed available RAM Prevents OOM kills, stable throughput
High-fidelity photography AVIF (SVT-AV1) Preserves gradients/textures, ~50% size reduction Moderate compute, significant CDN savings
Batch processing pipelines AVIF (libaom, effort 6) Maximum compression for archival/storage High CPU time, acceptable in async jobs

Configuration Template

# .env.production
IMAGE_PIPELINE_MODE=production
AVIF_ENCODER=svt-av1
AVIF_QUALITY=72
AVIF_EFFORT=5
WEBP_QUALITY=75
WEBP_EFFORT=4
MAX_CONCURRENT_AVIF=2
MAX_CONCURRENT_WEBP=10
QUEUE_BACKEND=redis
QUEUE_URL=redis://cache:6379
METRICS_ENDPOINT=http://monitoring:9090/metrics

// pipeline.config.ts
import dotenv from 'dotenv';
dotenv.config();

export const pipelineConfig = {
  avif: {
    encoder: (process.env.AVIF_ENCODER || 'svt-av1') as 'libaom' | 'svt-av1',
    quality: parseInt(process.env.AVIF_QUALITY || '72', 10),
    effort: parseInt(process.env.AVIF_EFFORT || '5', 10),
    concurrency: parseInt(process.env.MAX_CONCURRENT_AVIF || '2', 10)
  },
  webp: {
    quality: parseInt(process.env.WEBP_QUALITY || '75', 10),
    effort: parseInt(process.env.WEBP_EFFORT || '4', 10),
    concurrency: parseInt(process.env.MAX_CONCURRENT_WEBP || '10', 10)
  },
  queue: {
    backend: process.env.QUEUE_BACKEND || 'redis',
    url: process.env.QUEUE_URL || 'redis://localhost:6379'
  },
  metrics: {
    endpoint: process.env.METRICS_ENDPOINT || 'http://localhost:9090/metrics'
  }
};

Quick Start Guide

  1. Install dependencies: npm install sharp dotenv events
  2. Configure environment: Copy the .env.production template and adjust concurrency/quality values to match your infrastructure.
  3. Initialize the router: Import ImagePipelineRouter and pipelineConfig. Instantiate with new ImagePipelineRouter(pipelineConfig.avif.concurrency, pipelineConfig.webp.concurrency).
  4. Route workloads: Call router.routeAndEncode(buffer, 'static', jobId) for build assets, or 'dynamic' for user uploads. Listen to the metrics event for observability.
  5. Deploy with limits: Run AVIF workers in isolated containers with 3GB memory limits. Use a process manager or Kubernetes HPA to scale based on queue depth. Verify fallback chains in browser network panels before full rollout.