LLM function calling patterns

ai.chat.completions.create({ model: 'gpt-4o', messages, tools: registry.toOpenAITools(), tool_choice: 'auto', temperature: 0.1 });

const response = completion.choices[0].message; if (!response.tool_calls?.length) return [response];

const toolResults: OpenAI.ChatCompletionToolMessageParam[] = [];

for (const call of response.tool_calls) { const tool = registry.get(call.function.name); if (!tool) { toolResults.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify({ error: 'Tool not registered' }) }); continue; }

try {
  const parsed = tool.schema.parse(JSON.parse(call.function.arguments));
  const result = await tool.execute(parsed);
  toolResults.push({
    role: 'tool',
    tool_call_id: call.id,
    content: JSON.stringify(result)
  });
} catch (err) {
  toolResults.push({
    role: 'tool',
    tool_call_id: call.id,
    content: JSON.stringify({ error: 'Schema validation failed', details: (err as Error).message })
  });
}

}

return [response, ...toolResults]; }

### Architecture Rationale
- **Schema Validation at Boundary**: `zod` parsing occurs immediately after JSON deserialization. This prevents malformed data from reaching business logic.
- **Registry Pattern**: Centralizes tool metadata, enabling dynamic loading, A/B testing of tool descriptions, and consistent observability tagging.
- **Idempotent Execution**: Each tool call is isolated. Failures do not cascade. Results are formatted as tool messages for seamless continuation.
- **Low Temperature**: Function calling requires deterministic output. `temperature: 0.1` minimizes creative drift while preserving reasoning capability.
- **Separation of Concerns**: Generation, validation, routing, and execution are distinct phases. This enables independent testing, mocking, and scaling.

## Pitfall Guide

### 1. Overcomplicating Schemas with Deep Nesting
LLMs struggle with nested objects beyond two levels. Deep nesting increases token count and parsing failure rates.
**Best Practice**: Flatten schemas. Use string enums, explicit arrays, and top-level primitives. Reserve nesting for truly hierarchical data, and validate depth limits in CI.

### 2. Ignoring Tool Definition Token Limits
Verbose descriptions and excessive parameters consume context window budget. Long-running agents silently degrade when tool definitions exceed 15% of available tokens.
**Best Practice**: Cap tool descriptions at 80 characters. Use concise parameter names. Implement dynamic tool loading based on conversation state.

### 3. Synchronous Execution Blocking the Event Loop
Awaiting database queries or external APIs inside the tool executor without concurrency control causes latency spikes and timeout cascades.
**Best Practice**: Use `Promise.all` for independent calls. Implement circuit breakers for external dependencies. Set explicit timeouts per tool.

### 4. Missing Fallback and Repair Loops
When validation fails, naive implementations return errors to the LLM without guidance, causing infinite retry loops.
**Best Practice**: Return structured error messages with correction hints. Implement a max-retry threshold (typically 2). Fall back to a clarifying prompt if repair fails.

### 5. Tight Coupling Between Prompt and Tool Signatures
Embedding tool usage instructions directly in system prompts creates brittle integrations. Schema changes require prompt rewrites.
**Best Practice**: Decouple instructions from definitions. Use tool descriptions to convey usage. Maintain a versioned prompt registry separate from tool schemas.

### 6. No Observability or Tracing
Without request IDs, tool call IDs, and execution metrics, debugging production failures requires log scavenging.
**Best Practice**: Attach `trace_id` and `tool_call_id` to every execution. Log schema validation results, execution duration, and error types. Export to OpenTelemetry or equivalent.

### 7. Assuming Deterministic Outputs
LLMs do not guarantee field order, missing optional fields, or consistent enum casing. Treating responses as contract-compliant guarantees runtime crashes.
**Best Practice**: Always parse with a validator. Default missing optional fields explicitly. Normalize casing and whitespace before execution.

## Production Bundle

### Action Checklist
- [ ] Define all tool schemas using `zod` with explicit types, defaults, and constraints
- [ ] Implement a centralized tool registry with dynamic loading and versioning
- [ ] Add runtime validation immediately after JSON deserialization
- [ ] Set `tool_choice: 'auto'` and `temperature ≤ 0.2` for deterministic routing
- [ ] Attach `tool_call_id` and `trace_id` to all execution logs and metrics
- [ ] Implement max-retry logic with structured error hints for LLM repair
- [ ] Cap tool definition token usage below 15% of context window
- [ ] Test schemas against adversarial inputs and edge-case parameter combinations

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| Low-latency chatbot with single action | Single-Turn Structured Calling | Minimal overhead, fast validation, predictable routing | Low: $0.021/request |
| Complex data extraction with conditional logic | Multi-Turn Orchestrated Calling | Stateful reasoning, repair loops, context preservation | Medium: $0.034/request |
| High-volume batch processing | Batched Tool Execution with Queue | Parallelization, retry isolation, throughput scaling | Low: Amortized $0.015/request |
| Cost-constrained MVP | Prompt-Guided Function Calling + Schema Guard | Fastest integration, validation layer catches drift | Medium: Higher retry cost early |
| Multi-agent coordination | Registry-Based Tool Sharing + Event Bus | Decoupled execution, cross-agent observability | High: Infrastructure overhead |

### Configuration Template

```typescript
// tools/config.ts
import { z } from 'zod';
import { ToolRegistry } from './registry';
import { executeToolCall } from './orchestrator';
import OpenAI from 'openai';

export function initializeToolSystem() {
  const registry = new ToolRegistry();

  registry.register({
    name: 'get_weather',
    description: 'Fetch current weather and optional forecast',
    schema: z.object({
      location: z.string().min(2),
      unit: z.enum(['celsius', 'fahrenheit']).default('celsius'),
      include_forecast: z.boolean().default(false)
    }),
    execute: async (params) => {
      // Replace with actual API call
      return { temp: 22, unit: params.unit, forecast: params.include_forecast ? 'sunny' : null };
    }
  });

  registry.register({
    name: 'query_database',
    description: 'Execute read-only database query with filters',
    schema: z.object({
      table: z.string().regex(/^[a-z_]+$/),
      columns: z.array(z.string()).min(1),
      filters: z.record(z.string(), z.unknown()).optional(),
      limit: z.number().int().min(1).max(50).default(10)
    }),
    execute: async (params) => {
      // Replace with actual DB client
      return { rows: [], count: 0 };
    }
  });

  return { registry, executeToolCall };
}

// Usage in route/controller
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const { registry, executeToolCall: runTools } = initializeToolSystem();

export async function handleChatRequest(userMessage: string) {
  const messages: OpenAI.ChatCompletionMessageParam[] = [
    { role: 'system', content: 'You are a helpful assistant. Use tools when required.' },
    { role: 'user', content: userMessage }
  ];

  const updatedMessages = await runTools(openai, registry, messages);
  return updatedMessages;
}

Quick Start Guide

1. Install dependencies: npm install openai zod
2. Copy the Configuration Template into tools/config.ts and replace placeholder executors with actual API/DB clients.
3. Initialize the registry and pass it to executeToolCall alongside your OpenAI client and conversation history.
4. Validate responses by logging tool_call_id and execution duration. Add Sentry/OpenTelemetry for production tracing.
5. Deploy with temperature: 0.1 and tool_choice: 'auto'. Monitor schema validation failure rates and adjust constraints accordingly.