rt async function extractProduct(text: string): Promise<ProductExtraction> { const maxRetries = 3;

for (let attempt = 1; attempt <= maxRetries; attempt++) { try { const response = await openai.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'system', content: 'Extract product data from the provided text. Return strictly valid JSON matching the schema.' }, { role: 'user', content: text } ], // Enforce JSON structure response_format: { type: 'json_schema', json_schema: { name: 'product_extraction', schema: jsonSchema, strict: true, // Critical: Ensures strict adherence to schema }, }, });

  const content = response.choices[0]?.message?.content;
  if (!content) throw new Error('Empty response from model');

  // 1. Parse JSON
  const parsed = JSON.parse(content);

  // 2. Validate against Zod schema (runtime safety)
  const validationResult = ProductExtractionSchema.safeParse(parsed);
  
  if (!validationResult.success) {
    throw new Error(`Schema validation failed: ${validationResult.error.message}`);
  }

  return validationResult.data;

} catch (error) {
  // 3. Retry logic with error feedback
  if (attempt === maxRetries) {
    throw new Error(`Extraction failed after ${maxRetries} attempts: ${(error as Error).message}`);
  }
  
  // Inject error context into next attempt for self-correction
  console.warn(`Attempt ${attempt} failed: ${(error as Error).message}. Retrying...`);
  // In a full implementation, append the error message to the conversation history
  // or use a specialized retry wrapper.
}

}

throw new Error('Unreachable'); }

### Architecture Decisions

1.  **Strict Mode Enforcement:** Always enable `strict: true` (or equivalent) in API calls. This prevents the model from adding fields not defined in the schema, which can break downstream deserialization.
2.  **Decoupled Schema Definition:** Keep schemas in a shared module used by both the LLM integration and downstream services. This ensures type consistency across the entire data pipeline.
3.  **Validation Layer:** Never trust the LLM output implicitly. The `safeParse` step is non-negotiable. It catches semantic violations that syntax checks miss.
4.  **Error Feedback Loops:** Implement a mechanism to pass validation errors back to the model. If a field is missing or malformed, the retry prompt should include the specific validation error, allowing the model to self-correct.

## Pitfall Guide

### 1. Over-Constraining Enums and Formats
**Mistake:** Defining enums with hundreds of values or overly complex regex patterns for string formats.
**Impact:** Models struggle to select from large enum lists, increasing hallucination rates. Complex regex constraints may be ignored or cause generation stalls.
**Best Practice:** Keep enums under 20 items where possible. Use descriptive strings and validate formats in the Zod schema rather than relying on the model to generate perfect regex matches.

### 2. Nested Object Explosion
**Mistake:** Creating deeply nested schemas with multiple levels of optional arrays and objects.
**Impact:** Token consumption spikes, and models frequently omit nested fields or misalign JSON brackets.
**Best Practice:** Flatten schemas where feasible. If nesting is required, use `describe` annotations to guide the model. Test extraction on edge cases with missing nested data.

### 3. Ignoring Token Limits in Schemas
**Mistake:** Embedding massive JSON schemas in the prompt or system message without considering context window limits.
**Impact:** Truncation of the schema leads to partial enforcement. The model only sees part of the structure and generates invalid output.
**Best Practice:** Use API-level schema passing (e.g., `response_format.json_schema`) rather than embedding the schema in text. This optimizes token usage and ensures the model receives the full constraint.

### 4. Assuming Universal Structured Support
**Mistake:** Writing code that assumes all models support JSON mode or function calling equally.
**Impact:** Failures when switching to open-weight models or older API versions.
**Best Practice:** Abstract the structured output mechanism behind an interface. Implement fallbacks for models lacking native support, such as grammar-constrained decoding via vLLM or TGI, or regex extraction with aggressive validation.

### 5. Schema Drift in Production
**Mistake:** Updating the Zod schema without updating the prompt instructions or test cases.
**Impact:** The model continues generating old structures, or new fields are ignored.
**Best Practice:** Integrate schema changes into CI/CD. Run automated tests that verify LLM output against the current schema definition. Use versioned schemas if backward compatibility is required.

### 6. Hallucination of Required Fields
**Mistake:** Marking fields as required in the schema when the source text may not contain that information.
**Impact:** The model fabricates data to satisfy the schema, leading to data integrity issues.
**Best Practice:** Use `optional()` in Zod for fields that may be absent. In the system prompt, instruct the model to use `null` or omit fields when data is unavailable, and configure the schema to handle nulls gracefully.

### 7. Lack of Monitoring for Schema Violations
**Mistake:** Deploying structured extraction without monitoring validation failure rates.
**Impact:** Silent degradation of data quality. Teams remain unaware of rising error rates until downstream systems crash.
**Best Practice:** Instrument the validation layer to emit metrics on schema violation types. Set alerts for violation rate thresholds. Log failed validations for analysis and prompt refinement.

## Production Bundle

### Action Checklist

- [ ] **Define Zod/Pydantic Schema:** Create a strict schema definition file shared across services.
- [ ] **Implement JSON Schema Conversion:** Set up build-time or runtime conversion to JSON Schema.
- [ ] **Enable Strict Mode:** Configure API calls with `strict: true` and `response_format` constraints.
- [ ] **Add Validation Layer:** Wrap LLM responses in a `safeParse` or equivalent validation step.
- [ ] **Implement Retry with Feedback:** Build a retry mechanism that injects validation errors into subsequent attempts.
- [ ] **Flatten Complex Structures:** Review schemas for unnecessary nesting; flatten where possible.
- [ ] **Instrument Metrics:** Log schema violation rates, latency, and token usage per extraction call.
- [ ] **Test Edge Cases:** Create a test suite with inputs containing missing data, noise, and ambiguous entities.

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
| :--- | :--- | :--- | :--- |
| **High-Volume, Cost-Sensitive** | Native JSON Mode + Small Model | Low latency, minimal token overhead, high reliability with modern small models. | Low |
| **Complex Nested Extraction** | Function Calling + Large Model | Better handling of complex schemas and tool-use logic; reduces nesting errors. | Medium |
| **Open-Source/Local Models** | Grammar-Constrained Decoding (vLLM/TGI) | Enforces structure at the token level without provider lock-in. | Low (Infrastructure) |
| **Streaming Requirements** | JSON Mode + Streaming Parser | Allows incremental processing while maintaining structure; requires robust streaming JSON parser. | Low |
| **Legacy Model Support** | Few-Shot + Regex Fallback | Necessary when native structured output is unavailable; higher maintenance. | High |

### Configuration Template

**`schema/product.ts`**

```typescript
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

export const ProductSchema = z.object({
  id: z.string(),
  name: z.string(),
  price: z.number(),
  currency: z.string().default('USD'),
  tags: z.array(z.string()).optional(),
});

export const ProductJsonSchema = zodToJsonSchema(ProductSchema, {
  name: 'product',
  target: 'openApi',
});

export type Product = z.infer<typeof ProductSchema>;

clients/llm.ts

import { ProductJsonSchema } from './schema/product';

export const llmConfig = {
  model: 'gpt-4o-mini',
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'product_extraction',
      schema: ProductJsonSchema,
      strict: true,
    },
  },
  temperature: 0.1, // Low temperature for deterministic extraction
};

Quick Start Guide

1. Install Dependencies:

   npm install zod zod-to-json-schema openai
   

2. Define Schema: Create a Zod schema in schema.ts describing your target structure.
3. Convert Schema: Use zodToJsonSchema to generate the JSON Schema object.
4. Configure API Call: Pass the JSON Schema to the LLM client via response_format with strict: true.
5. Validate Output: Parse the response and run schema.safeParse() before using the data. Handle validation errors with a retry loop.

Structured output transforms LLMs from text generators into reliable data extraction engines. By enforcing contracts at the schema level and leveraging native API constraints, developers can eliminate parsing fragility, reduce latency, and build production-grade AI applications with confidence.