Model cards vs pre-registration: what counts as evidence under the EU AI Act

ty, violating the verifiability requirement. 4. Separation of Manifest and Result: The manifest contains the claim (what will be measured and the threshold). The result is generated separately. The verification step compares the claim against the result.

Implementation: TypeScript Pre-Commitment Engine

The following implementation demonstrates a PreCommitmentEngine that generates manifests and verifies claims. This code uses distinct interfaces and logic from any reference specifications, focusing on production-grade TypeScript patterns.

import { createHash } from 'crypto';

// Core interfaces for the manifest structure
interface EvaluationClaim {
  metric: string;
  threshold: number;
  comparator: 'gte' | 'lte' | 'eq';
  datasetDigest: string;
  seed: number;
  producerId: string;
  timestamp: string; // ISO 8601 of claim creation
}

interface VerificationResult {
  status: 'PASS' | 'FAIL' | 'TAMPERED';
  details: string;
}

class PreCommitmentEngine {
  /**
   * Generates a tamper-evident manifest for an evaluation claim.
   * Must be executed BEFORE the evaluation run.
   */
  public generateManifest(claim: EvaluationClaim): { manifest: EvaluationClaim; digest: string } {
    const canonicalBytes = this.toCanonicalBytes(claim);
    const digest = createHash('sha256').update(canonicalBytes).digest('hex');

    return {
      manifest: claim,
      digest,
    };
  }

  /**
   * Verifies a claim against a stored digest and checks the result.
   */
  public verifyClaim(
    storedDigest: string,
    currentClaim: EvaluationClaim,
    actualMetricValue: number
  ): VerificationResult {
    // Step 1: Check for tampering
    const currentBytes = this.toCanonicalBytes(currentClaim);
    const currentDigest = createHash('sha256').update(currentBytes).digest('hex');

    if (currentDigest !== storedDigest) {
      return {
        status: 'TAMPERED',
        details: `Digest mismatch. Expected ${storedDigest}, got ${currentDigest}. Claim may have been modified.`,
      };
    }

    // Step 2: Evaluate against threshold
    const meetsThreshold = this.evaluateThreshold(currentClaim, actualMetricValue);

    return {
      status: meetsThreshold ? 'PASS' : 'FAIL',
      details: meetsThreshold
        ? `Metric ${currentClaim.metric} value ${actualMetricValue} meets threshold ${currentClaim.threshold}.`
        : `Metric ${currentClaim.metric} value ${actualMetricValue} failed threshold ${currentClaim.threshold}.`,
    };
  }

  private toCanonicalBytes(obj: EvaluationClaim): Buffer {
    // Enforce sorted keys for deterministic serialization
    const sortedKeys = Object.keys(obj).sort();
    const sortedObj: Record<string, unknown> = {};
    for (const key of sortedKeys) {
      sortedObj[key] = obj[key as keyof EvaluationClaim];
    }
    return Buffer.from(JSON.stringify(sortedObj), 'utf-8');
  }

  private evaluateThreshold(claim: EvaluationClaim, value: number): boolean {
    switch (claim.comparator) {
      case 'gte': return value >= claim.threshold;
      case 'lte': return value <= claim.threshold;
      case 'eq': return value === claim.threshold;
      default: throw new Error(`Unknown comparator: ${claim.comparator}`);
    }
  }
}

Usage Example

const engine = new PreCommitmentEngine();

// 1. Define claim BEFORE running evaluation
const claim: EvaluationClaim = {
  metric: 'accuracy',
  threshold: 0.92,
  comparator: 'gte',
  datasetDigest: 'a1b2c3d4e5...', // Hash of the evaluation dataset
  seed: 42,
  producerId: 'sys-prod-01',
  timestamp: new Date().toISOString(),
};

const { manifest, digest } = engine.generateManifest(claim);

// 2. Store digest in immutable audit log immediately
// await auditLog.store({ digest, manifest, timestamp: Date.now() });

// 3. Run evaluation using manifest.seed and manifest.datasetDigest
// const result = runEvaluation(manifest);

// 4. Verify post-run
const verification = engine.verifyClaim(digest, manifest, 0.93);
// verification.status === 'PASS'

Rationale: The toCanonicalBytes method is critical. If the JSON serialization is not deterministic, the hash will change even if the claim content is identical, causing false TAMPERED results. The VerificationResult enum explicitly handles the TAMPERED state, which is required to detect post-hoc edits.

Pitfall Guide

Production implementations of pre-commitment patterns frequently encounter specific failure modes. The following pitfalls are derived from audit observations and engineering best practices.

1. Non-Canonical JSON Serialization

   • Explanation: Using a standard JSON serializer that does not sort keys. Two runs of the same claim produce different hashes due to key ordering variations in the runtime environment.
   • Fix: Implement explicit canonical serialization with sorted keys and consistent number formatting. Validate by hashing the same object twice and comparing digests.

2. Mutable Dataset References

   • Explanation: The manifest references a dataset by name or path (e.g., s3://bucket/eval-data.csv). If the dataset is updated, the evaluation results change, but the manifest remains valid.
   • Fix: Always include a cryptographic hash of the dataset content in the manifest. The verifier must hash the dataset used during the run and compare it to datasetDigest.

3. Seed Leakage or Omission

   • Explanation: The random seed is omitted from the manifest or is generated after the claim is created. This prevents deterministic re-derivation of results.
   • Fix: The seed must be part of the claim object and committed via the hash. The evaluation engine must read the seed from the manifest, not generate a new one.

4. Post-Hoc Threshold Tuning

   • Explanation: Engineers adjust the threshold after seeing initial results to achieve a "PASS" status. This is the exact behavior pre-commitment is designed to prevent.
   • Fix: Enforce policy that the threshold is immutable once the manifest is generated. If a threshold change is required, a new manifest must be generated, and the old run must be discarded or labeled as exploratory.

5. Storing Digest in Mutable Storage

   • Explanation: The digest is stored in the same database or file system as the claim. An attacker or accidental process could modify both the claim and the digest simultaneously.
   • Fix: Store the digest in a write-once audit log, a blockchain transaction, or a separate immutable storage system. The digest storage must be isolated from the claim storage.

6. Confusing Model Card with Manifest

   • Explanation: Teams attempt to embed the manifest hash inside the model card and treat the card as the evidence. This mixes human-readable prose with cryptographic evidence, complicating verification.
   • Fix: Maintain strict separation. The model card references the manifest ID or digest. The manifest is the evidence artifact. The model card is the summary artifact.

7. Ignoring Comparator Semantics

   • Explanation: The manifest specifies a threshold but omits the comparator (e.g., >= vs <=). For metrics like error rate, a lower value is better; for accuracy, a higher value is better. Ambiguity leads to verification errors.
   • Fix: Always include an explicit comparator field in the claim structure. Validate that the comparator matches the metric semantics during manifest generation.

Production Bundle

Action Checklist

• Define Claim Schema: Establish the mandatory fields for your evaluation claims (metric, threshold, comparator, dataset hash, seed, producer).
• Implement Canonicalizer: Build or integrate a JSON canonicalization library that enforces sorted keys and deterministic formatting.
• Create Manifest Generator: Develop a function that takes a claim, canonicalizes it, computes SHA-256, and returns the digest.
• Secure Digest Storage: Configure an immutable audit log or write-once storage for digests. Ensure isolation from claim data.
• Integrate with Eval Pipeline: Modify evaluation scripts to read parameters from the manifest and fail if the dataset hash does not match.
• Build Verification Service: Implement a verification endpoint that accepts a stored digest, a claim, and a result, returning PASS/FAIL/TAMPERED.
• Update Technical Documentation: Map the manifest generation process to Annex IV requirements and Article 15 verifiability in your compliance documentation.
• Train Engineering Teams: Conduct workshops on the difference between model cards and pre-registered manifests. Emphasize that manifests are evidence, not documentation.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High-Risk AI Deployment	Pre-Registered Manifest + Model Card	Regulatory requirement for verifiable accuracy. Model card provides context; manifest provides proof.	Medium. Requires engineering effort for manifest pipeline and secure storage.
Internal R&D / Prototyping	Model Card Only	No regulatory obligation. Speed of iteration is prioritized over verifiability.	Low. Minimal overhead.
Customer-Facing Transparency	Model Card with Manifest Reference	Builds trust by showing verifiable claims without exposing raw manifest data.	Low. Adds a reference link to existing documentation.
Audit Preparation	Manifest Verification Run	Proactively identifies tampered claims or missing digests before auditor review.	Low. One-time verification script execution.

Configuration Template

Use this JSON schema as the basis for your evaluation claims. This structure ensures all necessary fields are present for cryptographic binding and verification.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "EvaluationClaim",
  "type": "object",
  "required": [
    "metric",
    "threshold",
    "comparator",
    "datasetDigest",
    "seed",
    "producerId",
    "timestamp"
  ],
  "properties": {
    "metric": {
      "type": "string",
      "description": "Name of the metric (e.g., 'accuracy', 'f1_score')."
    },
    "threshold": {
      "type": "number",
      "description": "Target value for the metric."
    },
    "comparator": {
      "type": "string",
      "enum": ["gte", "lte", "eq"],
      "description": "Comparison operator for threshold evaluation."
    },
    "datasetDigest": {
      "type": "string",
      "description": "SHA-256 hash of the evaluation dataset content."
    },
    "seed": {
      "type": "integer",
      "description": "Random seed for deterministic evaluation."
    },
    "producerId": {
      "type": "string",
      "description": "Identifier of the system or team producing the claim."
    },
    "timestamp": {
      "type": "string",
      "format": "date-time",
      "description": "ISO 8601 timestamp of claim creation."
    }
  }
}

Quick Start Guide

1. Install Dependencies: Add crypto (Node.js built-in) and a JSON canonicalization library to your project.
2. Create Claim Object: Define your evaluation parameters in a JSON object matching the schema above.
3. Generate Digest: Run the canonicalization and hashing function to produce the SHA-256 digest.
4. Store Digest: Save the digest to your immutable audit log immediately.
5. Execute Evaluation: Run your evaluation using the seed and dataset hash from the claim.
6. Verify: Compare the result against the threshold and verify the digest matches the stored value. Output PASS, FAIL, or TAMPERED.

This workflow ensures that accuracy claims for high-risk AI systems are verifiable, tamper-evident, and compliant with the evidentiary standards of the EU AI Act.