TypeScript) are ideal choices due to their ecosystem support and runtime validation capabilities.
2. Validation Middleware: Validation must occur at the boundary between agents. The orchestrator intercepts the output of Agent A, validates it against Agent B's input schema, and only then invokes Agent B.
3. Error Handling Strategy: Validation failures must trigger explicit error states. Options include immediate abort, retry with a correction prompt, or fallback to a default safe state. Silent swallowing of errors is prohibited.
4. Versioning: Contracts must include a version field. This allows agents to evolve their schemas without breaking existing pipelines, enabling backward-compatible updates.
Implementation Example
The following TypeScript implementation demonstrates a type-safe orchestrator using Zod for schema validation. This approach ensures that both input and output structures are enforced at runtime.
import { z } from 'zod';
// 1. Define the Contract Interface
interface AgentContract<TInput extends z.ZodType, TOutput extends z.ZodType> {
name: string;
version: string;
inputSchema: TInput;
outputSchema: TOutput;
}
// 2. Define Specific Schemas for a Financial Analysis Agent
const RiskAnalysisInput = z.object({
holdings: z.array(z.string()).min(1, "Holdings list cannot be empty"),
timeframe: z.enum(['1d', '1w', '1m', '1y']),
riskTolerance: z.number().min(0).max(1),
});
const RiskAnalysisOutput = z.object({
valueAtRisk: z.number(),
volatilityIndex: z.number(),
alerts: z.array(z.object({
type: z.enum(['high_volatility', 'concentration_risk']),
severity: z.number().min(1).max(5),
message: z.string(),
})),
});
type RiskContract = AgentContract<typeof RiskAnalysisInput, typeof RiskAnalysisOutput>;
// 3. Agent Implementation
async function analyzeRisk(input: z.infer<typeof RiskAnalysisInput>): Promise<z.infer<typeof RiskAnalysisOutput>> {
// Simulate LLM call or external service
// In production, this would parse the LLM response and map it to the output shape
return {
valueAtRisk: 15420.50,
volatilityIndex: 0.85,
alerts: [
{ type: 'concentration_risk', severity: 3, message: 'Tech sector exposure > 40%' }
],
};
}
// 4. Orchestrator with Validation Middleware
class AgentOrchestrator {
private registry: Map<string, { fn: Function; contract: AgentContract<any, any> }> = new Map();
registerAgent<TInput extends z.ZodType, TOutput extends z.ZodType>(
contract: AgentContract<TInput, TOutput>,
implementation: (input: z.infer<TInput>) => Promise<z.infer<TOutput>>
) {
this.registry.set(contract.name, { fn: implementation, contract });
}
async execute<TInput extends z.ZodType, TOutput extends z.ZodType>(
agentName: string,
rawInput: unknown
): Promise<z.infer<TOutput>> {
const entry = this.registry.get(agentName);
if (!entry) throw new Error(`Agent ${agentName} not found`);
// Validate Input
const inputValidation = entry.contract.inputSchema.safeParse(rawInput);
if (!inputValidation.success) {
throw new Error(`Input validation failed for ${agentName}: ${inputValidation.error.message}`);
}
// Execute Agent
const rawOutput = await entry.fn(inputValidation.data);
// Validate Output
const outputValidation = entry.contract.outputSchema.safeParse(rawOutput);
if (!outputValidation.success) {
throw new Error(`Output schema violation in ${agentName}: ${outputValidation.error.message}`);
}
return outputValidation.data;
}
}
// 5. Usage
const orchestrator = new AgentOrchestrator();
const riskContract: RiskContract = {
name: 'risk_analyzer_v1',
version: '1.0.0',
inputSchema: RiskAnalysisInput,
outputSchema: RiskAnalysisOutput,
};
orchestrator.registerAgent(riskContract, analyzeRisk);
// Valid execution
const result = await orchestrator.execute('risk_analyzer_v1', {
holdings: ['AAPL', 'TSLA'],
timeframe: '1d',
riskTolerance: 0.05,
});
// This will throw: Output validation catches type mismatch
// e.g., if agent returns volatilityIndex as "high" instead of number
Rationale:
- Zod Integration: Using Zod provides compile-time type inference alongside runtime validation. This ensures that TypeScript types and runtime checks remain synchronized.
- Separation of Concerns: The agent function focuses solely on business logic. Schema validation is handled by the orchestrator, keeping agent code clean and testable.
- Explicit Errors: Validation failures throw descriptive errors including the schema path, enabling precise debugging.
Pitfall Guide
| Pitfall | Explanation | Fix |
|---|
| Over-Constraining Schemas | Defining schemas so rigid that the LLM cannot consistently satisfy them (e.g., requiring exact float precision). | Use ranges, tolerances, or optional fields where appropriate. Allow the LLM to return structured data that can be post-processed. |
| Ignoring Schema Versioning | Updating an agent's output schema without versioning breaks downstream consumers that expect the old structure. | Include a version field in the contract. Implement a compatibility layer or migration strategy for schema updates. |
| Validation Latency | Adding synchronous validation steps increases pipeline latency, especially with complex schemas. | Use async validation where possible. Cache schema compilation results. Profile validation overhead; for simple schemas, it is negligible. |
| Silent Fallbacks | Catching validation errors and returning default values without logging, masking systemic issues. | Always log validation failures with full context. Use fallbacks only for non-critical paths, and alert on fallback usage. |
| Schema Drift | The actual output of an agent diverges from its declared schema over time due to model updates or prompt changes. | Implement continuous integration checks that test agents against their schemas. Monitor validation failure rates in production. |
| Recursive Validation Loops | Agents that output data requiring validation by another agent, which then calls back, causing infinite loops. | Enforce a directed acyclic graph (DAG) structure for pipelines. Track execution depth and abort on cycles. |
| Treating Schema as Documentation | Defining schemas but not enforcing them at runtime, relying on developers to "follow the rules." | Make validation mandatory in the orchestrator. Reject any agent registration that lacks a contract. |
Production Bundle
Action Checklist
Decision Matrix
| Scenario | Recommended Approach | Why | Cost Impact |
|---|
| High-Risk Financial Pipeline | Strict Schema Validation + Immediate Abort | Zero tolerance for data corruption; safety is paramount. | Low (Validation is cheap; aborts save compute costs). |
| Exploratory Research Agent | LLM-as-Judge Validation + Retry | Flexibility needed; schema may evolve rapidly during experimentation. | Medium (Additional LLM calls for validation increase token costs). |
| High-Throughput Chatbot | Async Validation + Sampling | Latency sensitivity requires minimal overhead; sample validation to detect drift. | Low (Async validation has negligible latency impact). |
| Legacy System Integration | Adapter Pattern + Schema Mapping | Existing agents cannot be modified; wrap them with adapters that enforce contracts. | Medium (Development effort to build adapters). |
Configuration Template
Use this template to define agent contracts in a centralized configuration file. This supports declarative pipeline definition and easier maintenance.
agents:
- name: market_sentiment_engine
version: "2.1.0"
input:
type: object
required: [query, language]
properties:
query: { type: string, minLength: 1 }
language: { type: string, enum: [en, es, fr] }
max_tokens: { type: integer, minimum: 10, maximum: 2000 }
output:
type: object
required: [sentiment_score, key_themes]
properties:
sentiment_score: { type: number, minimum: -1.0, maximum: 1.0 }
confidence: { type: number, minimum: 0.0, maximum: 1.0 }
key_themes:
type: array
items: { type: string }
validation:
strict: true
retry_on_failure: 2
fallback: null
Quick Start Guide
- Install Dependencies: Add your schema validation library (e.g.,
zod for TypeScript or pydantic for Python) to your project.
- Define Your First Contract: Create a schema for a simple agent. Specify required fields, types, and constraints.
- Wrap the Agent: Modify your agent function to accept the validated input type and return the validated output type. Register it with the orchestrator using the contract.
- Run Validation Tests: Execute the agent with valid and invalid inputs. Verify that the orchestrator accepts valid data and rejects invalid data with clear error messages.
- Deploy: Integrate the orchestrator into your pipeline. Monitor validation metrics to ensure contracts are effective and adjust schemas as needed based on production behavior.