t:8080/v1', apiKey: process.env.PROVIDER_API_KEY, maxRetries: 2 };

### Step 2: Implement the Trusted Pricing Registry
Agents cannot be trusted to report their own costs. A malicious or misconfigured agent could claim a premium model call costs $0.01. The registry maintains authoritative pricing per tool/model, updated via provider rate cards or manual overrides.

```typescript
// pricing.registry.ts
export interface ToolPricing {
  toolName: string;
  costPerCall: number;
  tokenMultiplier?: number; // For token-based models
  currency: 'USD';
}

export class PricingRegistry {
  private registry: Map<string, ToolPricing> = new Map();

  constructor() {
    this.seedDefaults();
  }

  private seedDefaults(): void {
    this.registry.set('gpt-4o', { toolName: 'gpt-4o', costPerCall: 0.015, tokenMultiplier: 0.000005, currency: 'USD' });
    this.registry.set('claude-sonnet', { toolName: 'claude-sonnet', costPerCall: 0.003, tokenMultiplier: 0.000001, currency: 'USD' });
    this.registry.set('web_search', { toolName: 'web_search', costPerCall: 0.03, currency: 'USD' });
  }

  getActualCost(toolName: string, estimatedTokens?: number): number {
    const entry = this.registry.get(toolName);
    if (!entry) return 0;
    
    if (entry.tokenMultiplier && estimatedTokens) {
      return entry.costPerCall + (estimatedTokens * entry.tokenMultiplier);
    }
    return entry.costPerCall;
  }
}

Step 3: Build the Synchronous Budget Evaluator

Budget checks must be synchronous and atomic. Async checks in multi-agent loops cause race conditions where multiple concurrent calls pass the threshold before the ledger updates. We use a distributed lock (Redis or in-memory for single-node) to serialize budget mutations.

// cost.governor.ts
import { Request, Response, NextFunction } from 'express';
import { PricingRegistry } from './pricing.registry';
import { BudgetLedger } from './budget.ledger';

export class CostGovernor {
  private pricing: PricingRegistry;
  private ledger: BudgetLedger;

  constructor() {
    this.pricing = new PricingRegistry();
    this.ledger = new BudgetLedger();
  }

  middleware = async (req: Request, res: Response, next: NextFunction): Promise<void> => {
    const agentId = req.headers['x-agent-id'] as string;
    const toolName = req.headers['x-tool-name'] as string;
    const clientEstimate = parseFloat(req.headers['x-cost-estimate'] || '0');

    if (!agentId || !toolName) {
      res.status(400).json({ error: 'Missing governance headers' });
      return;
    }

    // 1. Verify agent exists and is active
    const agent = await this.ledger.getAgent(agentId);
    if (!agent) {
      res.status(401).json({ error: 'Unknown agent identity' });
      return;
    }
    if (agent.status === 'paused') {
      res.status(429).json({ error: 'Agent suspended due to budget exhaustion' });
      return;
    }

    // 2. Calculate authoritative cost (ignores client estimate)
    const estimatedTokens = parseInt(req.headers['x-token-count'] || '0');
    const actualCost = this.pricing.getActualCost(toolName, estimatedTokens);

    // 3. Atomic budget check with distributed lock
    const lockKey = `budget_lock:${agentId}`;
    const acquired = await this.ledger.acquireLock(lockKey, 2000);
    if (!acquired) {
      res.status(503).json({ error: 'Budget check rate-limited' });
      return;
    }

    try {
      const currentSpend = await this.ledger.getDailySpend(agentId);
      const dailyLimit = agent.budget.dailyLimit;

      if (currentSpend + actualCost > dailyLimit) {
        await this.ledger.pauseAgent(agentId);
        await this.ledger.logViolation(agentId, toolName, actualCost, currentSpend);
        res.status(429).json({ 
          error: 'Budget exceeded', 
          spent: currentSpend, 
          limit: dailyLimit,
          action: 'agent_paused'
        });
        return;
      }

      // 4. Commit spend and forward request
      await this.ledger.recordSpend(agentId, actualCost);
      await this.ledger.logTransaction(agentId, toolName, actualCost, 'registry');
      
      // Attach governance metadata for downstream observability
      req.governance = { approved: true, cost: actualCost, remaining: dailyLimit - (currentSpend + actualCost) };
      next();
    } finally {
      await this.ledger.releaseLock(lockKey);
    }
  };
}

Architecture Decisions & Rationale

• Synchronous Budget Mutation: Async budget checks fail under concurrent agent loops. Serialization via locks guarantees ledger consistency.
• Registry-Driven Pricing: Client-side estimates are untrustworthy. The registry acts as the source of truth, falling back to token multipliers for dynamic models.
• Header-Based Metadata: Passing x-agent-id, x-tool-name, and x-token-count via headers keeps the proxy framework-agnostic. The agent runtime injects these before forwarding.
• Graceful Degradation: When budgets are hit, the proxy returns 429 with structured metadata. Agents can catch this and trigger fallback behaviors (e.g., switch to cheaper models, request human approval, or terminate gracefully).

Pitfall Guide

1. Trusting Client-Side Cost Estimates

Explanation: Agents or SDK wrappers often calculate costs locally and pass them to the governance layer. A misconfigured agent or a prompt injection can report $0.01 for a $12.50 call. Fix: Never accept client estimates as authoritative. Maintain a server-side pricing registry that overrides all incoming cost claims. Use token counts only as a multiplier, not a flat fee.

2. Ignoring Token-Based Pricing Granularity

Explanation: Treating all model calls as flat-rate actions ignores the reality of token consumption. Long contexts, tool outputs, and multi-turn conversations drastically change actual spend. Fix: Implement token-aware pricing. Capture prompt_tokens and completion_tokens from provider responses, apply provider-specific multipliers, and log the breakdown. Update the registry when providers adjust rate cards.

3. Budget Window Misalignment

Explanation: Daily budgets work for batch jobs but fail for interactive agents that run in short bursts. Conversely, per-session budgets can be exhausted by a single runaway loop. Fix: Implement tiered budget windows: per-request (hard cap), per-session (rolling window), and per-day (aggregate). Allow agents to request budget extensions via human-in-the-loop approval workflows.

4. Race Conditions in Concurrent Agent Loops

Explanation: Multi-agent systems or parallel tool execution can trigger simultaneous budget checks. Without serialization, multiple calls pass the threshold before the ledger updates. Fix: Use distributed locks (Redis SETNX or database advisory locks) around budget mutations. Keep lock TTLs short (1-2 seconds) and implement retry logic with exponential backoff.

5. Missing Streaming Cost Aggregation

Explanation: Streaming responses deliver tokens incrementally. If cost is only calculated at the end of the stream, intermediate budget checks are inaccurate, and agents may overspend before the final tally. Fix: Hook into stream chunk events. Accumulate token counts in real-time, apply rolling cost estimates, and enforce soft limits that trigger warnings before hard caps are reached.

6. Hardcoded Thresholds Without Dynamic Scaling

Explanation: Static daily budgets don't account for workload variance. A research agent might need $50 on Monday but only $5 on Tuesday. Rigid limits cause unnecessary pauses or waste budget. Fix: Implement adaptive budgeting based on historical usage patterns. Use moving averages to set baseline limits, and allow temporary spikes during known high-intensity workflows.

7. Lack of Graceful Degradation Strategies

Explanation: When a budget is hit, agents often crash or hang, leaving workflows in an inconsistent state. Fix: Design fallback paths. On 429 responses, agents should switch to lower-cost models, cache intermediate results, request human escalation, or terminate with a structured summary. Never allow silent failures.

Production Bundle

Action Checklist

• Deploy governance proxy as a sidecar or dedicated service in your agent infrastructure
• Populate pricing registry with current provider rate cards and token multipliers
• Implement distributed locking for all budget mutation operations
• Inject x-agent-id and x-tool-name headers in your agent runtime before outbound calls
• Configure tiered budget windows (per-request, per-session, per-day)
• Set up real-time alerting for budget threshold breaches (80%, 95%, 100%)
• Implement graceful degradation handlers for 429 responses in agent logic
• Audit pricing registry monthly against provider updates and adjust multipliers

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Single-agent batch processing	Per-session budget with hard cap	Predictable workload; session boundaries align with task completion	Low (tight control prevents overruns)
Multi-agent collaborative workflow	Distributed proxy with tiered windows + dynamic scaling	Concurrent calls require atomic locks; workloads vary by phase	Medium (higher infra cost for locks/observability)
Interactive chatbot with streaming	Token-accumulating soft limits + fallback model switching	Streaming requires real-time tracking; user experience must be preserved	Low-Medium (reduces premium model overuse)
Research/Scraping agent with external APIs	Registry-enforced flat rates + daily aggregate cap	External tool costs are fixed; daily caps prevent runaway loops	Low (predictable per-action pricing)

Configuration Template

# governance.config.yaml
proxy:
  port: 8080
  log_level: info
  rate_limit:
    requests_per_second: 50
    burst: 10

budget:
  windows:
    - type: per_request
      max_cost: 5.00
      currency: USD
    - type: per_session
      max_cost: 25.00
      currency: USD
      ttl_minutes: 60
    - type: per_day
      max_cost: 100.00
      currency: USD
      reset_hour_utc: 0

registry:
  refresh_interval: 3600 # seconds
  fallback_strategy: use_client_estimate # only if registry miss
  providers:
    openai:
      models:
        gpt-4o: { base_cost: 0.015, token_multiplier: 0.000005 }
        gpt-4o-mini: { base_cost: 0.002, token_multiplier: 0.0000001 }
    anthropic:
      models:
        claude-sonnet-4-20250514: { base_cost: 0.003, token_multiplier: 0.000001 }

observability:
  metrics_endpoint: /metrics
  tracing:
    enabled: true
    sampler: 0.1
  alerts:
    - threshold: 0.8
      channel: slack
      message: "Agent {agent_id} approaching daily budget limit"
    - threshold: 1.0
      channel: pagerduty
      message: "Agent {agent_id} budget exhausted; auto-paused"

Quick Start Guide

1. Initialize the proxy service: Deploy the governance proxy using the provided configuration template. Ensure your agent runtime routes all provider calls through http://localhost:8080/v1.
2. Seed the pricing registry: Load current provider rate cards into the registry. Configure token multipliers for dynamic models and flat rates for external tools.
3. Inject governance headers: Modify your agent's HTTP client to attach x-agent-id, x-tool-name, and x-token-count to every outbound request.
4. Set budget thresholds: Define per-request, per-session, and per-day limits based on your workload profile. Enable soft-limit warnings at 80% to trigger fallback behaviors.
5. Validate enforcement: Run a controlled test with a known expensive tool call. Verify the proxy intercepts the request, applies registry pricing, deducts from the ledger, and returns 429 when the limit is crossed. Confirm agent graceful degradation triggers correctly.