Backend logging best practices

tematic approach covering format standardization, context propagation, security, and performance optimization. The following implementation uses TypeScript and pino, the industry-standard high-performance logger for Node.js, though the architectural principles apply to any backend stack.

1. Structured Logging with JSON Serialization

All logs must be emitted as JSON objects. This enables log aggregators (e.g., Elasticsearch, Datadog, Loki) to index fields efficiently.

Implementation:

import pino from 'pino';

// Define log level hierarchy
type LogLevel = 'fatal' | 'error' | 'warn' | 'info' | 'debug' | 'trace';

const LOG_LEVEL: LogLevel = (process.env.LOG_LEVEL as LogLevel) || 'info';

// Base logger configuration
export const createBaseLogger = () => {
  return pino({
    level: LOG_LEVEL,
    // Redaction ensures sensitive fields are masked before serialization
    redact: {
      paths: [
        'req.headers.authorization',
        'req.headers.cookie',
        'user.password',
        'user.creditCard',
        '*.token',
      ],
      censor: '***REDACTED***',
    },
    // Formatters for custom field injection
    formatters: {
      level(label) {
        return { level: label };
      },
      bindings(bindings) {
        return { service: bindings.serviceName || 'unknown' };
      },
    },
    // Timestamp configuration for standard ISO format
    timestamp: pino.stdTimeFunctions.isoTime,
  });
};

2. Context Propagation with Trace IDs

In distributed systems, a single user request spans multiple services. Logs must include a trace_id to correlate these spans. This requires middleware to inject the trace ID into the logger context.

Implementation:

import { Request, Response, NextFunction } from 'express';
import { v4 as uuidv4 } from 'uuid';
import { pino } from 'pino';

// Extend Request interface to include logger
declare global {
  namespace Express {
    interface Request {
      logger: pino.Logger;
    }
  }
}

export const traceIdMiddleware = (req: Request, _res: Response, next: NextFunction) => {
  // Extract trace ID from headers or generate new one
  const traceId = req.headers['x-trace-id'] as string || uuidv4();
  
  // Set header for downstream services
  req.headers['x-trace-id'] = traceId;

  // Create child logger bound to this request context
  req.logger = req.app.locals.logger.child({
    traceId,
    method: req.method,
    path: req.path,
    userId: req.user?.id, // Assuming auth middleware populates req.user
  });

  next();
};

Usage in Route Handler:

app.get('/api/users/:id', traceIdMiddleware, async (req, res) => {
  req.logger.info({ userId: req.params.id }, 'Fetching user profile');
  
  try {
    // Business logic
    const user = await userService.getById(req.params.id);
    req.logger.debug({ userId: req.params.id, found: true }, 'User retrieved');
    res.json(user);
  } catch (error) {
    req.logger.error({ err: error }, 'Failed to fetch user');
    res.status(500).json({ error: 'Internal Server Error' });
  }
});

3. Performance: Async Writing and Sampling

Synchronous logging blocks the event loop. pino writes asynchronously by default, but for high-throughput systems, implement sampling to reduce volume for non-critical endpoints.

Sampling Implementation:

import pino from 'pino';

const samplingLogger = pino({
  level: 'info',
  // Sample 1 out of 10 debug logs
  // Note: Sampling logic often depends on specific business rules
  // Here we demonstrate a custom transport or hook approach
  hooks: {
    logMethod(inputArgs, method) {
      // Example: Skip debug logs for health checks
      if (inputArgs[1]?.path === '/health' && inputArgs[0] === 'debug') {
        return;
      }
      return method.apply(this, inputArgs);
    },
  },
});

4. Architecture Decisions

• Library vs. Agent: Use a structured logging library in the application code to ensure fields are available at the source. Rely on sidecar agents (e.g., Fluent Bit) only for transport and enrichment, never for parsing unstructured text.
• Centralized Aggregation: Logs should stream to a centralized store. Local file rotation is insufficient for distributed debugging.
• Separation of Concerns: Application code should never import transport configurations. The logger instance is injected, keeping business logic decoupled from infrastructure details.

Pitfall Guide

Production logging failures rarely stem from a single bug; they accumulate from consistent anti-patterns. Avoid these common mistakes based on real-world failure analysis.

1. Logging PII and Secrets

Mistake: Logging request bodies, headers, or user objects without filtering. Impact: GDPR/CCPA violations, data breaches, and immediate security incidents. Best Practice: Implement a global redaction serializer. Never log raw request payloads. Explicitly allow-list fields for logging rather than black-listing secrets.

2. Synchronous I/O Blocking

Mistake: Using loggers that write synchronously to disk or network. Impact: Increased latency, thread starvation, and potential deadlocks under load. Best Practice: Use async loggers with internal buffers. Ensure the buffer flush strategy is configured to prevent data loss during crashes while maintaining non-blocking behavior.

3. Inconsistent Log Levels

Mistake: Using ERROR for expected business exceptions (e.g., "User not found") or INFO for debugging details. Impact: Alert fatigue. Error rate dashboards become noisy, masking actual system failures. Best Practice:

• ERROR: System failures requiring immediate attention (DB down, 5xx).
• WARN: Recoverable issues or deprecations (Rate limit hit, fallback used).
• INFO: Significant state changes (Request completed, Job started).
• DEBUG: Detailed flow for developers (Payload details, Cache misses).

4. Missing Context in Child Loggers

Mistake: Creating child loggers but failing to propagate essential context like trace_id or user_id. Impact: Logs become isolated events. Impossible to reconstruct request flow. Best Practice: Enforce context injection via middleware. Use TypeScript types to ensure req.logger always includes required fields.

5. High Cardinality in Log Fields

Mistake: Logging unique identifiers that vary per request (e.g., random session IDs) as top-level fields in metrics derived from logs. Impact: Metric explosion. Aggregation tools crash or become prohibitively expensive due to unique series creation. Best Practice: Reserve top-level fields for low-cardinality dimensions (service name, region, status code). Keep high-cardinality data (request IDs, user emails) in the log body for search, not metrics.

6. Log Injection Attacks

Mistake: Logging user input without sanitization. Impact: Attackers inject CRLF characters to forge log entries, potentially executing log injection attacks that mislead security audits or exploit parser vulnerabilities. Best Practice: Structured loggers like pino automatically escape special characters in JSON output. Avoid string interpolation for user input.

7. Ignoring Log Retention and Rotation

Mistake: Storing all logs indefinitely or losing logs due to lack of rotation. Impact: Storage costs spiral, or critical historical data is lost. Best Practice: Define retention policies based on compliance and utility. Use tiered storage: hot storage for recent logs, cold storage for archival. Implement lifecycle policies in your log aggregation platform.

Production Bundle

Action Checklist

• Enforce Structured Format: Replace all string-based logging with JSON serialization across all services.
• Inject Trace Context: Implement middleware to generate/propagate trace_id and bind it to the logger instance for every request.
• Configure PII Redaction: Define a redaction list in the logger configuration covering headers, tokens, and sensitive user fields.
• Standardize Log Levels: Audit existing logs and reclassify messages to align with the ERROR/WARN/INFO/DEBUG hierarchy.
• Implement Sampling: Apply sampling rules for high-volume endpoints (e.g., health checks, static assets) to reduce ingestion costs.
• Add Service Metadata: Ensure every log entry includes service.name, version, and environment tags.
• Set Up Alerting: Configure alerts on error rates and latency anomalies derived from logs, not just log volume.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Microservices Architecture	Distributed Tracing + Structured Logs	Correlates requests across service boundaries; essential for debugging.	Medium (Ingestion costs increase, but debugging time decreases).
High-Throughput API (>10k RPS)	Structured Logs + Aggressive Sampling	Prevents log volume from overwhelming the pipeline; maintains signal for errors.	Low (Sampling reduces ingestion costs by 60-80%).
Compliance-Heavy App (FinTech/Health)	Immutable Structured Logs + Full Retention	Audit trails require complete, tamper-evident records of all actions.	High (Storage costs increase; sampling may be restricted).
Legacy Monolith Migration	Strangler Fig Pattern for Logging	Gradually introduce structured logging in new components while wrapping legacy logs.	Low to Medium (Incremental effort; avoids big-bang rewrite).

Configuration Template

Copy this pino.config.ts for a production-ready logger setup in TypeScript.

// pino.config.ts
import pino from 'pino';
import pinoPretty from 'pino-pretty';

const isProduction = process.env.NODE_ENV === 'production';

const baseConfig = {
  level: process.env.LOG_LEVEL || 'info',
  redact: {
    paths: ['*.password', '*.secret', '*.token', 'req.headers.cookie'],
    censor: '***REDACTED***',
  },
  formatters: {
    level: (label: string) => ({ level: label }),
    bindings: (bindings: any) => ({
      service: process.env.SERVICE_NAME || 'backend',
      env: process.env.NODE_ENV || 'development',
      pid: bindings.pid,
    }),
  },
  timestamp: pino.stdTimeFunctions.isoTime,
};

export const logger = isProduction
  ? pino(baseConfig)
  : pino({
      ...baseConfig,
      transport: {
        target: 'pino-pretty',
        options: {
          colorize: true,
          translateTime: 'SYS:standard',
          ignore: 'pid,hostname',
        },
      },
    });

Quick Start Guide

1. Install Dependencies:

   npm install pino
   # For development only
   npm install -D pino-pretty
   

2. Create Logger Instance: Create src/logger.ts using the Configuration Template above. Export the logger instance.

3. Add Trace Middleware: Implement the traceIdMiddleware from the Core Solution section. Register it early in your Express/Fastify middleware stack.

4. Replace Console Logs: Update route handlers to use req.logger instead of console. Ensure errors are passed as objects: logger.error({ err }, 'Message').

5. Verify Output: Run the application and check logs. Ensure output is valid JSON in production and readable in development. Verify traceId appears in every log entry.

6. Deploy and Monitor: Deploy to staging. Verify logs are ingested correctly by your aggregation tool. Test searchability using traceId and level filters.