chitecture revolves around three components: a contract registry, a validation engine, and a routing orchestrator. Each agent declares its expected input and guaranteed output shapes. The orchestrator intercepts data between hops, validates against the registry, and only forwards payloads that pass structural checks.
Step 1: Define Typed Contracts
Contracts should be expressed as JSON Schema objects or TypeScript interfaces that map directly to runtime validation. Avoid embedding business logic inside the schema; keep it strictly structural. Use explicit types, enums, and range constraints to bound LLM output variability.
interface MarketAnalysisContract {
input: {
ticker: string;
window_days: number;
volatility_threshold: number;
};
output: {
trend_direction: 'bullish' | 'bearish' | 'neutral';
confidence_score: number;
risk_flags: string[];
};
}
Step 2: Implement Runtime Validation
The validation layer must run synchronously before data reaches the next agent. Use a schema compiler that converts definitions into fast validation functions. Pre-compiling schemas eliminates runtime parsing overhead and ensures consistent error reporting.
import Ajv from 'ajv';
class ContractValidator {
private ajv: Ajv;
private compiledSchemas: Map<string, Ajv.ValidateFunction>;
constructor() {
this.ajv = new Ajv({ allErrors: true, coerceTypes: true });
this.compiledSchemas = new Map();
}
register(schema: Record<string, unknown>, key: string): void {
const validate = this.ajv.compile(schema);
this.compiledSchemas.set(key, validate);
}
validate(payload: unknown, schemaKey: string): { valid: boolean; errors?: string[] } {
const validate = this.compiledSchemas.get(schemaKey);
if (!validate) {
throw new Error(`Unregistered schema: ${schemaKey}`);
}
const isValid = validate(payload);
if (!isValid) {
return {
valid: false,
errors: validate.errors?.map(e => `${e.instancePath} ${e.message}`)
};
}
return { valid: true };
}
}
Step 3: Orchestrate with Guardrails
The pipeline manager wraps agent execution in a validation loop. If an agent returns data that violates the contract, the system halts, logs the discrepancy, and optionally retries with a regeneration prompt. This prevents corrupted data from poisoning downstream agents.
class AgentPipeline {
constructor(
private validator: ContractValidator,
private registry: Map<string, (input: any) => Promise<any>>
) {}
async execute(stepSequence: string[], initialPayload: any): Promise<any> {
let currentData = initialPayload;
for (const stepId of stepSequence) {
const agentFn = this.registry.get(stepId);
if (!agentFn) throw new Error(`Agent ${stepId} not found`);
const rawOutput = await agentFn(currentData);
const validation = this.validator.validate(rawOutput, `${stepId}_out`);
if (!validation.valid) {
throw new ValidationError(
`Contract violation at ${stepId}: ${validation.errors?.join(', ')}`
);
}
currentData = rawOutput;
}
return currentData;
}
}
Architecture Decisions and Rationale
Why validate synchronously? Because asynchronous validation introduces race conditions and makes error attribution impossible. Synchronous checks guarantee that each hop is isolated and failures are caught immediately.
Why separate the registry from execution? Because it enables unit testing of agents against contracts without spinning up LLM instances. You can mock agent outputs, validate them against schemas, and verify pipeline logic deterministically.
Why use a dedicated schema compiler like AJV instead of native JSON.parse or manual type checks? Because native parsing lacks type enforcement, range checking, enum validation, and cross-field dependency resolution. Schema compilers provide optimized validation functions, detailed error paths, and consistent behavior across environments.
This design prioritizes reliability over raw throughput, which aligns with production requirements where correctness outweighs marginal latency gains. The validation overhead typically adds 2-5ms per hop, which is negligible compared to the cost of debugging cascading failures or reprocessing corrupted data.
Pitfall Guide
-
Over-Specifying Schemas
Explanation: Defining schemas that are too rigid (e.g., requiring exact string matches for open-ended analysis) causes constant validation failures when LLMs produce semantically correct but structurally different output.
Fix: Use additionalProperties: false cautiously. Allow optional fields and leverage oneOf or anyOf for flexible but bounded responses. Reserve strictness for critical numerical or categorical fields.
-
Validating Only at Pipeline Termination
Explanation: Checking contracts only after the final agent runs means errors propagate through multiple hops, making root-cause analysis nearly impossible.
Fix: Implement hop-by-hop validation. Each agent’s output must satisfy the next agent’s input contract before execution proceeds. Log validation results at every boundary.
-
Ignoring Schema Versioning
Explanation: As agent capabilities evolve, contracts change. Without versioning, updated agents break existing pipelines silently.
Fix: Namespace contracts with semantic versions (e.g., risk_assessor_v1, risk_assessor_v2). Maintain backward-compatible adapters during migration windows. Track schema changes in a dedicated changelog.
-
Confusing Structural Validation with Semantic Correctness
Explanation: A payload can pass JSON Schema validation but still contain logically invalid data (e.g., confidence_score: 1.5 when the contract expects 0-1).
Fix: Enforce numeric ranges, enum constraints, and cross-field dependencies within the schema. Use minimum, maximum, and pattern keywords aggressively. Layer domain validators on top of structural contracts.
-
Hardcoding Contracts in Agent Logic
Explanation: Embedding schema definitions inside agent functions couples generation logic with validation rules, making refactoring difficult.
Fix: Externalize contracts into a dedicated registry module. Agents should only declare their contract key, not the schema itself. This enables centralized updates and consistent validation across environments.
-
Neglecting Fallback Strategies
Explanation: When validation fails, halting the pipeline is safe but not always practical. Production systems need graceful degradation.
Fix: Implement retry-with-correction loops. If validation fails, inject the schema error back into the LLM prompt and request regeneration before failing hard. Cap retries to prevent infinite loops.
-
Assuming JSON Schema Covers All Edge Cases
Explanation: JSON Schema cannot validate business logic, temporal consistency, or domain-specific constraints.
Fix: Use a two-phase validation approach. Phase 1: structural validation via JSON Schema. Phase 2: semantic validation via custom business rules. This keeps schemas clean while enforcing domain accuracy.
Production Bundle
Action Checklist
Decision Matrix
| Scenario | Recommended Approach | Why | Cost Impact |
|---|
| High-frequency trading agents | Strict schema + synchronous validation | Zero tolerance for malformed data; latency acceptable | Higher compute cost for validation, lower risk cost |
| Creative content generation pipeline | Lenient schema + semantic fallback | Output variability expected; strictness causes false failures | Lower validation overhead, higher manual review cost |
| Multi-step research workflow | Versioned contracts + hop-by-hop validation | Complex chains require isolation of failure points | Moderate infrastructure cost, significantly faster debugging |
| Prototype / MVP | Dynamic schema inference + post-execution validation | Speed of iteration prioritized over reliability | Lowest initial cost, highest technical debt risk |
Configuration Template
{
"contracts": {
"data_ingestor_v1": {
"input": {
"source_url": { "type": "string", "format": "uri" },
"parse_mode": { "type": "string", "enum": ["raw", "cleaned"] }
},
"output": {
"extracted_text": { "type": "string", "minLength": 1 },
"metadata": {
"type": "object",
"properties": {
"word_count": { "type": "integer", "minimum": 0 },
"language": { "type": "string", "pattern": "^[a-z]{2}$" }
},
"required": ["word_count", "language"]
}
}
},
"summarizer_v1": {
"input": {
"text": { "type": "string", "minLength": 50 },
"target_length": { "type": "integer", "minimum": 50, "maximum": 500 }
},
"output": {
"summary": { "type": "string" },
"key_points": { "type": "array", "items": { "type": "string" }, "minItems": 3 }
}
}
},
"orchestration": {
"validation_mode": "strict",
"retry_on_failure": true,
"max_retries": 2,
"fallback_strategy": "regenerate_with_schema_hint",
"error_logging": {
"enabled": true,
"include_payload_snippet": true,
"redact_sensitive_fields": ["api_key", "token"]
}
}
}
Quick Start Guide
- Extract Current Outputs: Run your existing agents and log raw outputs. Identify format inconsistencies, missing fields, and token waste. Map these to expected downstream requirements.
- Draft Initial Contracts: Create JSON Schema definitions for each agent’s expected input and guaranteed output. Start with structural constraints only. Avoid business logic in this phase.
- Integrate Validator: Add a validation middleware to your pipeline runner. Intercept outputs, run them through the schema compiler, and block non-compliant payloads. Log validation results at every hop.
- Test in Isolation: Run agents against their contracts using synthetic inputs. Verify that valid data passes and invalid data triggers clear, actionable errors. Mock LLM calls to speed up testing.
- Deploy with Monitoring: Enable validation failure logging. Track mismatch rates, adjust schema strictness based on real-world drift, and implement retry loops before enforcing hard halts. Review logs weekly to refine constraints.