The Bot Left a Fingerprint: Detecting and Attributing LLM-Generated Passwords

equently detected, accounting for 7,951 of the 28,000 classified instances, followed by Qwen (6,643) and Google (3,184). Anthropic outputs also showed the highest detection confidence at 92%, indicating a very distinct statistical profile.

Core Solution

Detecting LLM-generated secrets requires moving beyond simple entropy checks. The most effective approach is Markov Chain analysis, which models the probability of character transitions. Since LLMs generate text token-by-token based on conditional probabilities, the resulting passwords exhibit non-random transition patterns that Markov models can capture with high accuracy.

Architecture Design

A robust detection system should implement a three-tiered analysis pipeline:

1. Model-Specific Chains: Trained on individual model outputs to identify the exact source. Achieves ~55% attribution accuracy.
2. Provider-Specific Chains: Aggregates data across a provider's model family. Achieves ~65% attribution accuracy.
3. Generic LLM Chain: Trained on the aggregate dataset of all LLM passwords. Used to calculate perplexity; LLM passwords typically exhibit perplexity levels half that of truly random strings.

Classification Logic: A password is flagged as LLM-generated if:

• A model-specific chain predicts it with >75% confidence.
• A provider-specific chain predicts it with >75% confidence.
• The generic chain calculates a perplexity score <100.

Note: Models like xAI are often excluded from generic detection because they generate human-like weak passwords (e.g., P@ssw0rd...) that overlap with poor human practices, increasing false positives.

Implementation Example

The following TypeScript implementation demonstrates a SecretFingerprintEngine that builds transition matrices and scores secrets against trained profiles. This differs from standard regex scanners by analyzing character transition probabilities.

interface TransitionMatrix {
  [currentChar: string]: Map<string, number>;
}

interface AnalysisResult {
  isLikelyLLM: boolean;
  confidence: number;
  topProvider: string | null;
  perplexity: number;
}

class SecretFingerprintEngine {
  private profiles: Map<string, TransitionMatrix> = new Map();
  private genericProfile: TransitionMatrix | null = null;

  /**
   * Trains a profile using a corpus of known LLM-generated secrets.
   * Builds a Markov chain representing character transition probabilities.
   */
  trainProfile(profileId: string, corpus: string[]): void {
    const matrix: TransitionMatrix = {};
    const totalTransitions: Map<string, number> = new Map();

    for (const secret of corpus) {
      for (let i = 0; i < secret.length - 1; i++) {
        const current = secret[i];
        const next = secret[i + 1];

        if (!matrix[current]) {
          matrix[current] = new Map();
        }

        const count = matrix[current].get(next) || 0;
        matrix[current].set(next, count + 1);

        const total = totalTransitions.get(current) || 0;
        totalTransitions.set(current, total + 1);
      }
    }

    // Normalize to probabilities
    for (const [char, transitions] of Object.entries(matrix)) {
      const total = totalTransitions.get(char) || 1;
      for (const [nextChar] of transitions) {
        transitions.set(nextChar, transitions.get(nextChar)! / total);
      }
    }

    this.profiles.set(profileId, matrix);
  }

  /**
   * Analyzes a secret against all trained profiles.
   * Returns the best match and a perplexity score.
   */
  analyze(secret: string): AnalysisResult {
    let bestConfidence = 0;
    let bestProvider: string | null = null;
    let totalLogProb = 0;
    let transitionCount = 0;

    // Score against specific profiles
    for (const [profileId, matrix] of this.profiles) {
      const score = this.calculateLogProbability(secret, matrix);
      if (score > bestConfidence) {
        bestConfidence = score;
        bestProvider = profileId;
      }
    }

    // Calculate generic perplexity if available
    if (this.genericProfile) {
      totalLogProb = this.calculateLogProbability(secret, this.genericProfile);
      transitionCount = secret.length - 1;
    }

    const perplexity = transitionCount > 0 
      ? Math.exp(-totalLogProb / transitionCount) 
      : Infinity;

    return {
      isLikelyLLM: bestConfidence > 0.75 || perplexity < 100,
      confidence: bestConfidence,
      topProvider: bestProvider,
      perplexity
    };
  }

  private calculateLogProbability(secret: string, matrix: TransitionMatrix): number {
    let logProb = 0;
    for (let i = 0; i < secret.length - 1; i++) {
      const current = secret[i];
      const next = secret[i + 1];
      const prob = matrix[current]?.get(next) || 0.001; // Smoothing
      logProb += Math.log(prob);
    }
    return logProb;
  }
}

Rationale for Choices:

• Transition Normalization: Converting counts to probabilities ensures the model is invariant to corpus size and focuses on relative likelihoods.
• Smoothing: Adding a small epsilon (0.001) prevents zero probabilities for unseen transitions, which would otherwise skew results.
• Perplexity Metric: Perplexity measures how "surprised" the model is by the input. Lower perplexity indicates the secret fits the LLM distribution well.
• Modular Profiles: Separating model and provider profiles allows for granular attribution while maintaining a fallback generic detector.

Pitfall Guide

1. The Uniqueness Fallacy

   • Explanation: Assuming that because an LLM generates unique passwords across samples, they are secure. As shown with GPT-5, 100% uniqueness can coexist with 52% substring overlap.
   • Fix: Evaluate secrets based on entropy and pattern distribution, not just uniqueness. Implement Markov-based detection to catch structural biases.

2. Agent Hardcoding Blindspots

   • Explanation: Security teams often scan human commits but ignore AI agent activity. Agents can autonomously generate and commit secrets to Terraform files or JSON configs without human review.
   • Fix: Configure secret scanners to intercept AI agent hooks. Ensure pre-commit checks run on all commits, regardless of author metadata.

3. Provider Leakage Risk

   • Explanation: Requesting a password from an LLM sends the request and response through the provider's infrastructure. The secret may be logged, cached, or used in future training, compromising it before it reaches the developer.
   • Fix: Never use LLMs for secret generation. Use local Cryptographically Secure Pseudo-Random Number Generators (CSPRNG) or dedicated password managers.

4. Threshold Drift

   • Explanation: LLM providers update models frequently. A detection threshold calibrated for an older model version may produce false positives or miss new patterns.
   • Fix: Continuously retrain detection profiles with fresh samples. Implement automated regression testing for the fingerprinting engine against new model releases.

5. Context Ignorance

   • Explanation: Flagging every password that matches an LLM pattern can lead to alert fatigue, especially if human-generated passwords occasionally match common substrings.
   • Fix: Combine statistical detection with context analysis. Prioritize alerts for secrets found in .env files, configuration templates, or commits co-authored by AI agents.

6. Attack Surface Expansion

   • Explanation: The same Markov chains used for detection can be weaponized by attackers to generate high-probability password candidates, cracking LLM-generated secrets faster than brute-force.
   • Fix: Rotate any secrets suspected of being LLM-generated immediately. Treat LLM-generated secrets as compromised by default.

Production Bundle

Action Checklist

• Audit AI Prompts: Review team guidelines and AI agent configurations to prohibit password generation requests.
• Deploy Fingerprint Scanner: Integrate a Markov-based detection tool into CI/CD pipelines to scan for LLM signatures.
• Configure Agent Guardrails: Enable secret scanning on AI coding agent hooks to prevent autonomous hardcoding.
• Rotate Suspected Keys: Identify and rotate credentials in .env files, Terraform states, and connection strings that match LLM patterns.
• Educate Developers: Train teams on the difference between LLM output and cryptographic randomness; mandate CSPRNG usage.
• Monitor Velocity: Track the rate of LLM-generated secret commits; a spike may indicate a new model release or workflow change.
• Validate Detection Models: Periodically test the scanner against the latest model versions to ensure accuracy.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High-Volume CI/CD	Regex + Entropy Check	Fast execution, low resource usage; catches obvious weak patterns.	Low
Suspected AI Leak	Markov Chain Analysis	High accuracy for LLM fingerprints; detects subtle biases missed by entropy.	Medium
AI Agent Monitoring	Hook-Based Scanner	Real-time prevention; stops secrets before commit.	Low
Forensic Audit	Perplexity Scoring	Quantifies deviation from randomness; useful for attribution.	Medium
Compliance Requirement	CSPRNG Enforcement	Guarantees cryptographic strength; satisfies audit standards.	Low

Configuration Template

Use this TypeScript configuration to initialize the fingerprint engine with baseline profiles. In production, these profiles should be generated from a curated corpus of model outputs.

// fingerprint.config.ts
import { SecretFingerprintEngine } from './SecretFingerprintEngine';

const engine = new SecretFingerprintEngine();

// Load training corpora (paths to datasets of known LLM passwords)
const trainingCorpora = {
  'anthropic-claude': './data/corpus/anthropic_claude.txt',
  'openai-gpt': './data/corpus/openai_gpt.txt',
  'meta-llama': './data/corpus/meta_llama.txt',
  'generic-llm': './data/corpus/aggregate_llm.txt',
};

// Initialize profiles
for (const [profileId, filePath] of Object.entries(trainingCorpora)) {
  const corpus = loadCorpusFromFile(filePath); // Implementation dependent
  engine.trainProfile(profileId, corpus);
  
  // Set generic profile for perplexity calculation
  if (profileId === 'generic-llm') {
    engine.setGenericProfile(corpus);
  }
}

// Detection thresholds
export const DETECTION_CONFIG = {
  confidenceThreshold: 0.75,
  perplexityThreshold: 100,
  excludeProviders: ['xAI'], // Models with human-like weak passwords
};

export { engine };

Quick Start Guide

1. Install Dependencies: Set up the TypeScript environment and import the SecretFingerprintEngine class.
2. Generate Training Data: Collect a sample of passwords from target LLM models (e.g., 200 samples per model) to build accurate profiles.
3. Initialize Scanner: Load the training corpora into the engine and configure detection thresholds based on your risk tolerance.
4. Integrate into Pipeline: Add a pre-commit hook or CI step that runs the scanner against changed files, flagging secrets with high LLM confidence.
5. Review and Rotate: Investigate alerts, verify if secrets are LLM-generated, and rotate any compromised credentials immediately.