s.config.taskName}

Status: COMPILED

Generated: ${new Date().toISOString()}

Source Runs: ${this.executionHistory.length}

Target Configuration

• Endpoint: ${this.extractEndpoint(stableTrace)}
• Required Headers: Referer, Accept: application/json
• Pagination: batch_size=50, offset_parameter=page

Navigation Sequence

1. Initialize session with geolocation override
2. Query category enum: ${this.extractCategory(stableTrace)}
3. Execute batch fetch with pagination loop
4. Validate response schema before persistence

Error Recovery

• If 403: Rotate Referer header and retry once
• If timeout: Reduce batch_size to 25
• If schema mismatch: Fallback to DOM parsing

Performance Baseline

• Avg Cost: $${stableTrace.tokenCost.toFixed(2)}

• Avg Latency: ${stableTrace.latencyMs}ms `;

  writeFileSync(blueprintPath, content); console.log([Compiler] Blueprint written to: ${blueprintPath}); return blueprintPath; }

  private extractEndpoint(trace: ExecutionTrace): string { return trace.steps.find(s => s.includes('api.')) || 'unknown_endpoint'; }

  private extractCategory(trace: ExecutionTrace): string { return trace.steps.find(s => s.includes('category=')) || 'default'; } }

// Usage Example async function main() { const compiler = new WorkflowCompiler({ maxIterations: 5, stabilityThreshold: 0.05, outputDir: './compiled_skills', taskName: 'craigslist_search' });

await compiler.runConvergenceLoop(async () => { // Simulate agent execution against live target return { runId: crypto.randomUUID(), steps: ['init_session', 'query_api', 'paginate_results', 'validate_schema'], errors: [], latencyMs: 28000, tokenCost: 0.12, success: true }; }); }

main().catch(console.error);

### Architecture Decisions and Rationale

1. **Markdown Artifacts Over JSON/YAML**: The compiled workflow uses markdown because it remains human-readable, diff-friendly in version control, and requires zero schema validation overhead. AI models parse structured markdown natively without additional tokenization steps.
2. **Convergence Criteria Based on Variance**: Stability is measured through cost and latency variance across the last three runs, not arbitrary iteration counts. This prevents premature graduation and ensures the workflow has genuinely stabilized.
3. **Separation of Compilation and Execution**: The compiler runs only during the discovery phase. Production pipelines load pre-compiled blueprints, eliminating iterative overhead from live traffic.
4. **Explicit Error Recovery Paths**: The blueprint documents fallback procedures rather than relying on the model to improvise during failures. This reduces token consumption during error states and improves predictability.

## Pitfall Guide

### 1. Iterating on Static Content
**Explanation**: Applying the convergence loop to fixed HTML catalogs or server-rendered pages with stable schemas wastes tokens and execution budget. The agent will repeatedly rediscover the same DOM structure without finding optimization opportunities.
**Fix**: Run a lightweight schema detection pass first. If the target returns consistent HTML with predictable selectors, bypass the loop and use a static parser (e.g., BeautifulSoup, Cheerio). Reserve iteration for JS-heavy, gated, or undocumented surfaces.

### 2. Blind Trust in Undocumented Endpoints
**Explanation**: The convergence loop frequently surfaces hidden JSON APIs or internal routes. These endpoints lack SLAs, may change without notice, and often bypass official rate limits.
**Fix**: Treat discovered endpoints as provisional. Implement response schema validation, cache control headers, and fallback routing to official APIs or DOM parsing. Log endpoint stability metrics and trigger re-compilation if failure rates exceed thresholds.

### 3. Ignoring Target Site Drift
**Explanation**: Web applications update frequently. A compiled blueprint becomes stale when selectors change, pagination parameters shift, or authentication flows are modified.
**Fix**: Implement drift detection by monitoring execution success rates and latency spikes. Schedule periodic re-compilation during low-traffic windows. Version blueprints and maintain rollback paths to previous stable states.

### 4. Confusing Context Window with Cross-Run Memory
**Explanation**: Teams often assume larger context windows solve persistence. A million-token window only extends single-session reasoning depth; it does not survive process termination.
**Fix**: Architect for explicit state persistence. Use compiled artifacts, external knowledge bases, or vector stores for cross-run memory. Treat context windows as working memory, not long-term storage.

### 5. Over-Engineering the Artifact Format
**Explanation**: Attempting to serialize workflows into complex schemas, binary formats, or database tables adds parsing overhead and breaks human auditability.
**Fix**: Stick to structured markdown. Include clear sections for configuration, sequence, error handling, and performance baselines. Keep the artifact under 2KB to minimize token consumption during load.

### 6. Missing Convergence Criteria
**Explanation**: Running the loop indefinitely or stopping after a fixed iteration count leads to either wasted budget or unstable blueprints.
**Fix**: Define quantitative stability thresholds (cost variance, latency variance, success rate). Require a minimum of three consecutive stable runs before graduation. Implement early termination if metrics degrade.

### 7. Lack of Execution Fallbacks
**Explanation**: Blueprints assume the happy path. When network conditions change or target behavior shifts, agents following rigid instructions fail without recovery.
**Fix**: Embed conditional branches in the blueprint. Document primary and secondary approaches. Implement a lightweight validator that checks target responsiveness before executing the compiled sequence.

## Production Bundle

### Action Checklist
- [ ] Audit target sites for static vs. dynamic content before deploying the convergence loop
- [ ] Define quantitative convergence thresholds (cost variance < 5%, latency variance < 10%, 100% success rate)
- [ ] Implement drift detection monitoring on compiled blueprints with automated re-compilation triggers
- [ ] Validate all discovered undocumented endpoints with schema checks and fallback routing
- [ ] Version control all compiled artifacts and maintain rollback capabilities
- [ ] Budget token consumption separately for compilation phase vs. execution phase
- [ ] Implement lightweight target health checks before loading blueprints in production runs
- [ ] Document error recovery paths explicitly in the artifact rather than relying on model improvisation

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| Static HTML catalog with fixed schema | Static parser (Cheerio/BeautifulSoup) | No discovery overhead; deterministic extraction | ~$0.01/run |
| JS-heavy SPA with undocumented APIs | Convergence loop → Compiled blueprint | Iteration surfaces hidden endpoints and pagination logic | Compilation: ~$2-5/run; Execution: ~$0.12/run |
| Gated portal with auth flows | Hybrid: Compile auth sequence, parse content | Reduces token burn on repeated authentication | Compilation: ~$3/run; Execution: ~$0.24/run |
| High-frequency data feed (>10k runs/day) | API-first or official endpoint | Compiled blueprints still incur load overhead; native APIs are cheaper | ~$0.005/run (if available) |
| Rapidly changing target site | Scheduled re-compilation + drift monitoring | Blueprints decay quickly; automation prevents stale execution | Compilation: ~$2/run; Execution: ~$0.15/run |

### Configuration Template

```typescript
// compiler.config.ts
export const compilationConfig = {
  taskName: 'federal_grants_portal',
  outputDir: './compiled_skills',
  maxIterations: 5,
  stabilityThreshold: 0.05,
  convergenceMetrics: {
    maxCostVariance: 0.05,
    maxLatencyVariance: 0.10,
    requiredSuccessRate: 1.0,
    minStableRuns: 3
  },
  driftDetection: {
    enabled: true,
    checkInterval: '24h',
    failureThreshold: 0.15,
    autoRecompile: true
  },
  security: {
    validateUndocumentedEndpoints: true,
    requireSchemaValidation: true,
    fallbackToDomParsing: true
  }
};

// blueprint_loader.ts
export class BlueprintExecutor {
  async loadAndExecute(blueprintPath: string, targetUrl: string) {
    if (!this.validateBlueprint(blueprintPath)) {
      throw new Error('Blueprint validation failed. Trigger re-compilation.');
    }
    
    const blueprint = this.parseMarkdown(blueprintPath);
    return this.executeCompiledSteps(blueprint, targetUrl);
  }

  private validateBlueprint(path: string): boolean {
    // Check file existence, version compatibility, and drift indicators
    return existsSync(path);
  }

  private parseMarkdown(path: string) {
    // Extract sections: Configuration, Sequence, Error Recovery, Performance
    return { /* structured object */ };
  }

  private async executeCompiledSteps(blueprint: any, target: string) {
    // Execute documented sequence with embedded fallbacks
    return { success: true, latency: 27000, cost: 0.12 };
  }
}

Quick Start Guide

1. Install the convergence package: Add Browserbase's open-source skills plugin to your agent environment via the Claude Agent SDK marketplace (/plugin marketplace add browserbase/skills).
2. Configure compilation parameters: Set target task name, output directory, iteration limits, and stability thresholds in your compiler configuration file.
3. Run the discovery phase: Execute the convergence loop against your target site. Monitor iteration logs until cost and latency variance stabilize across three consecutive runs.
4. Validate the compiled artifact: Review the generated markdown blueprint for accuracy, verify undocumented endpoints with schema checks, and test execution against a staging environment.
5. Deploy to production: Replace stateless agent calls with blueprint loader invocations. Enable drift detection monitoring and schedule periodic re-compilation during maintenance windows.