The conversation you saved to disk still has the SSN in it

Zero-Trace Logging: Architecting PII-Safe LLM Conversation Storage

Current Situation Analysis

Large Language Model (LLM) applications ingest unstructured, free-form text. Users naturally include sensitive data in prompts: Social Security numbers for identity verification, credit card details for billing inquiries, or medical codes for health advice. While the model processes this input transiently, engineering teams routinely persist conversation histories to disk for debugging, audit trails, and model fine-tuning.

The persistence layer is frequently treated as a secondary concern. Developers often implement naive serialization loops—dumping message arrays directly to JSONL files using standard library functions. This creates a silent data liability. The raw PII enters the log file exactly as the user typed it.

This oversight is systemic because the threat model is misaligned. Teams focus on securing the inference endpoint and the database, assuming logs are ephemeral or internal. Security audits routinely uncover JSONL repositories containing unredacted PII. The complexity compounds when persistence logic is scattered across multiple modules; adding a sanitization hook requires refactoring every write path.

Compliance frameworks (GDPR, PCI-DSS, HIPAA) emphasize data minimization and protection at rest. Storing raw PII in logs, even with access controls, violates the principle of least privilege and increases the blast radius of any storage breach. The industry standard is shifting toward write-time sanitization, where sensitive data is transformed before it ever touches the storage medium.

WOW Moment: Key Findings

The critical architectural decision is when sanitization occurs. Most teams default to read-time filtering, storing raw data and applying masks during retrieval. This approach fails under audit scrutiny because the raw data exists on disk. Write-time sanitization eliminates the raw data from the storage layer entirely.

Strategy	Disk Content	Breach Exposure	Compliance Defense	Historical Flexibility
Read-Time Filter	Raw PII	High: Raw data exists; filter can be bypassed or misconfigured.	Weak: "We stored it but filtered access." Auditors reject this for regulated data.	High: Original data preserved; rules can be re-applied.
Write-Time Sanitization	Masked Data	Low: Only masked values on disk. Breach yields no sensitive info.	Strong: "We never stored the raw value." Meets data minimization requirements.	Low: Irreversible. Original data is lost after write.
Write-Time + Encryption	Ciphertext	Negligible: Requires key compromise + decryption to access masked data.	Strongest: Defense-in-depth. Satisfies strict encryption-at-rest mandates.	Low: Irreversible; key rotation required for access changes.

Why this matters: Write-time sanitization shifts the security boundary. You accept irreversibility to guarantee that the storage layer cannot become a source of data leakage. For regulated industries, this is often the only acceptable pattern. The loss of historical flexibility is a calculated trade-off for audit safety.

Core Solution

The solution requires a persistence abstraction that intercepts message lists before serialization. This component must support a pluggable sanitization strategy and optional encryption. The architecture follows a pipeline pattern: Sanitize → Serialize → Encrypt → Write.

Architecture Decisions

1. Sanitization Interface: The sanitizer must be a callable or strategy object. This decouples the detection logic (regex, NLP, third-party redactors) from the I/O layer.
2. Write-Time Execution: Sanitization runs synchronously during the write operation. This ensures the file system never receives raw payloads.
3. Encryption Composition: Encryption is applied after sanitization. The ciphertext contains the masked data, not the original. This allows safe storage even if the sanitization logic has gaps, though gaps should be minimized.
4. Atomic Writes: To prevent corruption, writes should use a temporary file and atomic rename. This ensures readers never see partial JSONL files.
5. Immutability: The sanitizer must not mutate input dictionaries in place. It should return new objects to prevent side effects in the application's in-memory state.

Implementation Example

The following TypeScript implementation demonstrates the pattern. It defines a SecureLogCodec that handles serialization, sanitization, and encryption.

import { createCipheriv, randomBytes, createDecipheriv } from 'crypto';
import { writeFileSync, renameSync, unlinkSync, readFileSync } from 'fs';
import { join } from 'path';

// Interface for sanitization strategy
interface Sanitizer {
  sanitize(role: string, content: string): string;
}

// Example Regex Sanitizer
class RegexSanitizer implements Sanitizer {
  private patterns: RegExp[];

  constructor(patterns: RegExp[]) {
    this.patterns = patterns;
  }

  sanitize(role: string, content: string): string {
    // Only sanitize user inputs to preserve assistant context
    if (role !== 'user') return content;
    
    let sanitized = content;
    for (const pattern of this.patterns) {
      sanitized = sanitized.replace(pattern, '[REDACTED]');
    }
    return sanitized;
  }
}

// Secure Log Codec
class SecureLogCodec {
  private filePath: string;
  private sanitizer: Sanitizer;
  private encryptionKey?: Buffer;

  constructor(
    filePath: string,
    sanitizer: Sanitizer,
    encryptionKey?: string
  ) {
    this.filePath = filePath;
    this.sanitizer = sanitizer;
    if (encryptionKey) {
      this.encryptionKey = Buffer.from(encryptionKey, 'hex');
    }
  }

  persist(messages: Array<{ role: string; content: string }>): void {
    // 1. Sanitize messages (Immutable transform)
    const sanitizedMessages = messages.map(msg => ({
      ...msg,
      content: this.sanitizer.sanitize(msg.role, msg.content)
    }));

    // 2. Serialize to JSONL
    const jsonlContent = sanitizedMessages
      .map(msg => JSON.stringify(msg))
      .join('\n');

    // 3. Encrypt if key provided
    let payload: Buffer | string = jsonlContent;
    if (this.encryptionKey) {
      const iv = randomBytes(16);
      const cipher = createCipheriv('aes-256-cbc', this.encryptionKey, iv);
      const encrypted = Buffer.concat([
        iv,
        cipher.update(jsonlContent, 'utf8'),
        cipher.final()
      ]);
      payload = encrypted;
    }

    // 4. Atomic Write
    const tempPath = `${this.filePath}.tmp`;
    writeFileSync(tempPath, payload);
    renameSync(tempPath, this.filePath);
  }

  load(): Array<{ role: string; content: string }> {
    const raw = readFileSync(this.filePath);
    
    // Decrypt if necessary
    let jsonlContent: string;
    if (this.encryptionKey) {
      const iv = raw.subarray(0, 16);
      const data = raw.subarray(16);
      const decipher = createDecipheriv('aes-256-cbc', this.encryptionKey, iv);
      jsonlContent = Buffer.concat([
        decipher.update(data),
        decipher.final()
      ]).toString('utf8');
    } else {
      jsonlContent = raw.toString('utf8');
    }

    // Parse JSONL
    return jsonlContent
      .split('\n')
      .filter(line => line.trim() !== '')
      .map(line => JSON.parse(line));
  }
}

// Usage
const sanitizer = new RegexSanitizer([
  /\b\d{3}-\d{2}-\d{4}\b/g, // SSN pattern
  /\b\d{16}\b/g             // Credit card pattern
]);

const codec = new SecureLogCodec(
  './logs/session_42.jsonl',
  sanitizer,
  process.env.LOG_ENCRYPTION_KEY
);

const conversation = [
  { role: 'user', content: 'My SSN is 123-45-6789 and card is 4111111111111111.' },
  { role: 'assistant', content: 'I have masked your details. How can I help?' },
  { role: 'user', content: 'Please update my address.' }
];

codec.persist(conversation);
const loaded = codec.load();

console.log(loaded[0].content);
// Output: "My SSN is [REDACTED] and card is [REDACTED]."

Pitfall Guide

1. The Read-Time Illusion

Explanation: Storing raw data and applying filters during read operations. This leaves PII on disk, accessible to anyone with file system access or backup privileges. Fix: Enforce write-time sanitization. The persistence layer must never accept or store raw PII.

2. Key Co-Location

Explanation: Storing encryption keys in the same directory or configuration file as the encrypted logs. If an attacker accesses the storage, they gain both data and keys. Fix: Use a Key Management Service (KMS), environment variables injected at runtime, or a secrets manager. Keys must never be persisted alongside data.

3. The Assistant Echo Trap

Explanation: Sanitizing only user messages while ignoring assistant responses. If the assistant echoes user PII (e.g., "Your SSN 123-45-6789 is verified"), the PII leaks into the log. Fix: Implement context-aware sanitization. Sanitize assistant responses if they contain patterns matching known PII, or sanitize all roles if the model is prone to echoing.

4. Atomicity Failure

Explanation: Writing directly to the target file. If the process crashes mid-write, the file contains partial JSONL, causing parse errors for readers. Fix: Write to a temporary file, then use an atomic rename operation. This ensures readers always see a complete file or nothing.

5. Regex Limitations

Explanation: Relying solely on regex for PII detection. Regex misses variations, typos, or obfuscated PII. False negatives leave data exposed. Fix: Combine regex with NLP-based redaction tools for high-risk fields. Accept that no sanitizer is perfect; encryption provides a safety net for residual risk.

6. In-Place Mutation

Explanation: The sanitizer modifies the input message objects directly. This corrupts the in-memory state of the application, causing downstream logic to see redacted data unexpectedly. Fix: The sanitizer must return new objects. Use spread operators or immutable update patterns to preserve the original conversation state.

7. Cross-Message Context Leaks

Explanation: Sanitizing messages individually misses PII that spans multiple turns. For example, a user says "My name is John" in turn 1 and "My SSN is 123" in turn 7. A single-message sanitizer might miss the association, though the SSN is still caught. However, complex PII might require context. Fix: Acknowledge this limitation. For critical data, use batch sanitization if available, or ensure the sanitizer is robust enough to catch isolated PII regardless of context.

Production Bundle

Action Checklist

• Define Sanitization Policy: Document which PII types must be masked and the acceptable replacement tokens.
• Implement Sanitizer Strategy: Create a sanitizer class supporting regex and/or NLP detection. Ensure immutability.
• Configure Secure Codec: Initialize the persistence codec with the sanitizer and optional encryption key.
• Secure Key Storage: Provision encryption keys via KMS or secrets manager. Verify keys are not in source control.
• Test Breach Scenario: Simulate a file system breach. Verify that extracted logs contain only masked data and that decryption fails without the key.
• Add Error Handling: Implement logging for sanitization failures or encryption errors. Fail closed if sanitization cannot be guaranteed.
• Rotate Keys: Establish a key rotation schedule. Plan for re-encryption of historical logs if required.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Regulated Data (PCI/HIPAA)	Write-Time Sanitization + Encryption	Compliance requires data minimization and encryption at rest. Raw data cannot exist on disk.	High: Requires KMS, strict key management, and robust sanitization testing.
Internal Debugging Only	Write-Time Sanitization (No Encryption)	Reduces breach risk while avoiding encryption overhead. Sufficient for non-regulated internal logs.	Low: Minimal overhead. Sanitizer logic is the primary cost.
Need Historical Replay	Write-Time Sanitization + Separate Raw Store	If raw data is needed for re-training, store it in a separate, highly secured vault with distinct access controls.	High: Dual storage architecture. Increases complexity and cost.
High-Volume Streaming	Batch Sanitization + Append Mode	Full overwrite is inefficient for long sessions. Batch processing reduces CPU overhead.	Medium: Requires custom append logic and batch sanitizer integration.

Configuration Template

// config/log-secure-codec.ts
import { SecureLogCodec } from './secure-log-codec';
import { RegexSanitizer } from './sanitizers/regex-sanitizer';
import { NlpSanitizer } from './sanitizers/nlp-sanitizer';
import { CompositeSanitizer } from './sanitizers/composite-sanitizer';

// Define patterns for high-confidence matches
const regexPatterns = [
  /\b\d{3}-\d{2}-\d{4}\b/g,
  /\b\d{16}\b/g,
  /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi
];

// Initialize sanitizers
const regexSanitizer = new RegexSanitizer(regexPatterns);
const nlpSanitizer = new NlpSanitizer({ 
  models: ['ssn', 'credit_card', 'email', 'phone'] 
});

// Composite sanitizer runs regex first for speed, then NLP for coverage
const sanitizer = new CompositeSanitizer([regexSanitizer, nlpSanitizer]);

// Load encryption key from secure source
const encryptionKey = process.env.VAULT_LOG_KEY;

if (!encryptionKey) {
  throw new Error('Encryption key missing. Logs will be unencrypted.');
}

// Export configured codec instance
export const conversationVault = new SecureLogCodec(
  process.env.LOG_STORAGE_PATH || './logs/conversations.jsonl',
  sanitizer,
  encryptionKey
);

Quick Start Guide

1. Install Dependencies: Ensure crypto and fs modules are available. Install any NLP redaction libraries if using advanced sanitization.
2. Define Sanitizer: Create a sanitizer class implementing the sanitize interface. Configure patterns or models for your PII types.
3. Initialize Codec: Instantiate SecureLogCodec with your file path, sanitizer, and encryption key.
4. Persist Conversations: Call persist(messages) before writing logs. The codec handles sanitization, serialization, encryption, and atomic writes automatically.
5. Load Logs: Call load() to retrieve messages. The codec decrypts and parses the JSONL transparently.