Catch docs-to-code drift in TypeScript PRs before merge

on

The solution relies on a CLI tool designed for TypeScript repositories that analyzes the relationship between documentation anchors and code files. The tool operates by reading explicit mappings, traversing the local TypeScript import/export graph, and generating a traceability report. This process requires no external services, AI inference, or code uploads; everything runs locally within the repository context.

Implementation Workflow

1. Define Documentation Anchors: Embed unique identifiers in documentation files. These anchors serve as the source of truth for claims.
2. Configure Mappings: Create a configuration file that links anchors to TypeScript seed files. These seed files act as entry points for graph traversal.
3. Execute Analysis: Run the CLI to scan the repository, resolve imports, and verify mappings against the current codebase.
4. Review Report: Inspect the generated report in the CI workflow summary or artifacts to identify violations and coverage gaps.

Code Examples

Documentation Anchor Definition

Embed anchors in Markdown or YAML documentation using a consistent naming convention. The anchor should be descriptive and unique.

## SPEC.AUTH.JWT_EXPIRY
JWT tokens must expire after 15 minutes.
The expiration logic is enforced by the authentication middleware.

Traceability Configuration

Create a configuration file (e.g., spec-trace.yaml) to define the scope and mappings. The configuration specifies which files to analyze and links anchors to entry points.

version: 1
scope:
  include:
    - src/**/*.ts
  exclude:
    - src/**/*.test.ts
    - src/**/*.spec.ts

mappings:
  SPEC.AUTH.JWT_EXPIRY:
    entry_points:
      - src/auth/jwt-validator.ts
      - src/config/security-policies.ts

  SPEC.API.RATE_LIMITING:
    entry_points:
      - src/middleware/rate-limiter.ts

CI Integration

Integrate the analysis into the GitHub Actions workflow. The tool generates a report in the job summary and uploads artifacts for detailed inspection.

jobs:
  traceability-check:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Dependencies
        run: npm ci

      - name: Run AnchorMap Analysis
        uses: fstepho/anchormap-action@v1
        with:
          config: spec-trace.yaml
          fail_on_violation: true
          baseline: .anchormap-baseline.json

Report Output

The tool produces a structured report indicating the status of the analysis, summary metrics, and any violations found.

{
  "report_id": "TRACE-2024-05-20",
  "status": "VIOLATION_DETECTED",
  "summary": {
    "total_anchors": 12,
    "mapped_anchors": 10,
    "orphaned_anchors": 2,
    "stale_mappings": 0,
    "code_coverage": "68%"
  },
  "violations": [
    {
      "type": "ORPHANED_SPEC",
      "anchor": "SPEC.AUTH.JWT_EXPIRY",
      "message": "No entry points defined in configuration."
    },
    {
      "type": "UNTRACED_FILE",
      "file": "src/utils/crypto-helpers.ts",
      "message": "File is within scope but not reachable from any mapped anchor."
    }
  ],
  "suggested_actions": [
    "Add entry points for SPEC.AUTH.JWT_EXPIRY in spec-trace.yaml.",
    "Review src/utils/crypto-helpers.ts for potential inclusion in mappings."
  ]
}

Architecture Decisions

• Explicit Mappings: The tool requires manual configuration of mappings rather than inferring them. This design choice prioritizes auditability and precision. While it introduces maintenance overhead, it ensures that reviewers can see exactly which code paths are intended to satisfy each claim.
• Local Graph Traversal: The tool analyzes the TypeScript import/export graph locally. It resolves static imports to determine which files are reachable from the seed files. This avoids the latency and privacy concerns associated with remote analysis services.
• Report-Driven Feedback: Results are delivered via workflow summaries and artifacts rather than inline PR comments. This approach reduces noise and provides a comprehensive view of the traceability state without requiring additional permissions.
• Policy Enforcement: The tool supports configurable policies, such as failing the build on unmapped anchors or stale mappings. This allows teams to enforce documentation standards as part of the merge criteria.

Pitfall Guide

Adopting explicit traceability requires careful planning to avoid common pitfalls. The following issues frequently arise during implementation and can be mitigated with best practices.

1. Granularity Mismatch

   • Explanation: Mapping a broad documentation claim to a large file that contains multiple unrelated features can lead to false confidence. The claim may be satisfied by only a small portion of the file, but the tool reports the entire file as covered.
   • Fix: Use specific entry points where possible. If a file contains multiple features, consider splitting the file or using more granular anchors that map to specific functions or classes.

2. Stale Mappings

   • Explanation: When code is refactored or moved, mappings may point to files that no longer exist or no longer contain the relevant logic. The tool will flag these as stale, but teams often ignore these warnings.
   • Fix: Treat stale mapping warnings as high-priority issues. Integrate regular sweeps into the development workflow to update mappings whenever code structure changes.

3. Anchor Naming Inconsistency

   • Explanation: Inconsistent anchor naming conventions make it difficult to manage mappings and search for claims. This can lead to duplicate anchors or unmapped claims due to typos.
   • Fix: Establish a strict naming convention, such as DOMAIN.FEATURE.CLAIM. Enforce this convention through documentation guidelines and CI checks.

4. Ignoring Degraded Analysis

   • Explanation: The tool may encounter issues resolving the TypeScript graph, such as circular dependencies or dynamic imports. This results in degraded analysis, where the report is incomplete but still generated.
   • Fix: Monitor for degraded analysis warnings. Refactor code to eliminate circular dependencies and prefer static imports where possible to improve graph resolution.

5. Over-Indexing Documentation

   • Explanation: Attempting to map every sentence in the documentation creates excessive maintenance overhead and dilutes the focus on critical claims.
   • Fix: Prioritize mapping high-value claims, such as API contracts, security requirements, and behavioral specifications. Leave descriptive text unmapped unless it directly impacts implementation.

6. Confusing Traceability with Correctness

   • Explanation: The tool verifies that a claim is linked to code, but it does not verify that the code correctly implements the claim. A mapped claim may still be buggy or incomplete.
   • Fix: Use traceability as a complement to testing, not a replacement. Ensure that mapped code paths are covered by unit and integration tests to validate correctness.

7. Scope Misconfiguration

   • Explanation: Including test files or generated code in the analysis scope can skew coverage metrics and produce irrelevant findings.
   • Fix: Carefully configure the scope to include only production code. Exclude test files, generated artifacts, and third-party dependencies.

Production Bundle

This section provides actionable resources for implementing traceability in a production environment.

Action Checklist

• Define Anchor Convention: Establish a naming convention for documentation anchors (e.g., DOMAIN.FEATURE.CLAIM) and document it in the contribution guidelines.
• Create Configuration File: Initialize spec-trace.yaml with the repository scope and initial mappings for critical claims.
• Map High-Value Claims: Prioritize mapping API contracts, security requirements, and behavioral specifications. Avoid mapping descriptive text.
• Integrate CI Pipeline: Add the AnchorMap action to the GitHub Actions workflow. Configure policies to fail on violations if appropriate.
• Set Baseline: Generate an initial baseline artifact to track changes over time. Update the baseline after resolving known issues.
• Review Reports: Inspect traceability reports in the workflow summary. Address unmapped anchors and stale mappings promptly.
• Monitor Coverage: Track code coverage metrics to ensure that mapped anchors reach a significant portion of the codebase.

Decision Matrix

Use this matrix to determine when to adopt explicit traceability and which configuration options to use.

Scenario	Recommended Approach	Why	Cost Impact
Regulated API	Strict Policy + Full Mapping	Auditability and compliance require verifiable links between specs and code.	High maintenance, low risk.
Rapid Prototyping	Loose Policy + Selective Mapping	Speed is prioritized; only map critical claims to avoid overhead.	Low maintenance, medium risk.
Legacy Refactor	Baseline Tracking + Gradual Mapping	Track drift during refactoring; update mappings incrementally.	Medium maintenance, low risk.
Small Team	Manual Review + Linting	Overhead of explicit mapping may not justify benefits for small codebases.	Low maintenance, high risk.

Configuration Template

Copy this template to initialize traceability in a TypeScript repository. Adjust the scope and mappings to match your project structure.

version: 1

# Define the scope of files to analyze
scope:
  include:
    - src/**/*.ts
  exclude:
    - src/**/*.test.ts
    - src/**/*.spec.ts
    - src/generated/**

# Map documentation anchors to code entry points
mappings:
  # Example: Security requirement
  SPEC.AUTH.JWT_EXPIRY:
    entry_points:
      - src/auth/jwt-validator.ts
      - src/config/security-policies.ts

  # Example: API contract
  SPEC.API.RATE_LIMITING:
    entry_points:
      - src/middleware/rate-limiter.ts

  # Example: Behavioral specification
  SPEC.DATA.CACHING_STRATEGY:
    entry_points:
      - src/services/cache-manager.ts

Quick Start Guide

Follow these steps to set up traceability in under five minutes.

1. Install CLI: Add the AnchorMap CLI to your project dependencies or install it globally.

   npm install --save-dev anchormap-cli
   

2. Initialize Config: Create a spec-trace.yaml file in the repository root using the template above.
3. Add Anchor: Insert an anchor into a documentation file.

   ## SPEC.AUTH.JWT_EXPIRY
   JWT tokens must expire after 15 minutes.
   

4. Run Scan: Execute the analysis command to generate a report.

   npx anchormap scan --config spec-trace.yaml
   

5. Inspect Report: Review the output in the terminal or CI summary. Address any violations by updating mappings or documentation.