.build())
        .toList();

    return McpToolProvider.builder()
        .mcpClients(serviceClients)
        .build();
}

@Bean
public DynamicAgentInterface orchestrationAgent(McpToolProvider toolResolver) {
    return AiServices.builder(DynamicAgentInterface.class)
        .chatModel(primaryLanguageModel)
        .toolProvider(toolResolver)
        .maxSequentialToolsInvocations(6)
        .build();
}

}

**Architecture Rationale**:
- `toolProvider` replaces static `.tools()` registration. The framework handles schema serialization and runtime dispatch automatically.
- `maxSequentialToolsInvocations` is a critical safety valve. It caps the number of tool calls the LLM can chain in a single reasoning turn. Without it, ambiguous prompts can trigger infinite invocation loops.
- The provider abstracts service ownership. The agent never sees URLs or network topology; it only interacts with function signatures.

### Step 3: Runtime Execution Flow

When a user submits a query, the execution pipeline follows a deterministic cycle:

1. **Schema Injection**: LangChain4j calls `tools/list` on each registered MCP server during agent initialization. Schemas are serialized into `functionDeclaration` objects and injected into the system prompt.
2. **Reasoning & Selection**: The LLM parses the user query against available tool descriptions and generates a `functionCall` payload with resolved arguments.
3. **Framework Interception**: LangChain4j intercepts the call, routes it to the correct MCP client, and executes an HTTP POST to the `/mcp/message` endpoint.
4. **Service Execution**: The target service processes the request, runs the underlying business logic, and returns a structured result.
5. **Response Mapping**: The framework wraps the result in a `functionResponse` envelope and feeds it back to the LLM for final answer generation.

This cycle repeats until the LLM determines the query is satisfied or hits the sequential invocation limit. Independent calls within a chain can execute concurrently when virtual threads are enabled, significantly reducing total wall-clock time.

## Pitfall Guide

### 1. Unbounded Tool Invocation Chains
**Explanation**: LLMs will aggressively chain tools when given open-ended prompts. Without explicit limits, a single query can trigger dozens of sequential calls, exhausting rate limits and inflating latency.
**Fix**: Always configure `maxSequentialToolsInvocations`. Set conservative limits (3-6) based on expected workflow complexity. Monitor invocation counts in production and adjust dynamically if needed.

### 2. Synchronous HTTP Blocking
**Explanation**: Each MCP tool call is an HTTP request. In a traditional thread pool, sequential chains block worker threads, causing thread starvation under concurrent load.
**Fix**: Enable Spring Boot virtual threads. They allow the framework to park waiting I/O operations without consuming OS threads, enabling true parallel execution for independent calls.

### 3. Ambiguous Tool Signatures
**Explanation**: The LLM relies entirely on parameter names and descriptions to resolve arguments. Vague naming (`getData`, `processRequest`) or missing type hints causes misrouted calls or invalid payloads.
**Fix**: Enforce strict naming conventions (`getStockByProduct`, `calculateFraudRisk`). Include parameter constraints in descriptions (e.g., `threshold: integer (1-100)`). Validate schemas against OpenAPI-style strictness before deployment.

### 4. Silent Service Degradation
**Explanation**: When an MCP server returns HTTP 5xx or times out, the framework propagates the error as a function result. The LLM may retry indefinitely or hallucinate a fallback response.
**Fix**: Implement circuit breaker patterns at the transport layer. Configure fallback prompts that instruct the agent to skip unavailable tools gracefully. Log degradation events separately from normal tool calls.

### 5. Mixing JVM-Local and Remote Tools Incorrectly
**Explanation**: Developers often register local `@Tool` methods alongside MCP providers in the same agent. This creates inconsistent execution paths and complicates testing.
**Fix**: Establish a strict architectural boundary. Use `@Tool` exclusively for same-JVM operations (e.g., sub-agent delegation, local data transformations). Reserve MCP providers for cross-service or external API interactions.

### 6. Ignoring Tool Call Rate Limits
**Explanation**: Downstream services often enforce strict rate limits. An agent chaining multiple tools can easily exceed thresholds, triggering 429 responses that degrade user experience.
**Fix**: Implement client-side throttling or request queuing. Use Resilience4j or Spring Cloud Circuit Breaker to enforce concurrency limits. Add exponential backoff for retryable HTTP status codes.

### 7. Over-Fetching Tool Schemas
**Explanation**: Registering all available tools for every agent increases prompt token consumption and confuses the LLM with irrelevant capabilities.
**Fix**: Implement tool namespace filtering. Create agent-specific provider instances that only expose capabilities relevant to their domain. Use metadata tags in MCP server definitions to enable selective discovery.

## Production Bundle

### Action Checklist
- [ ] Define explicit `maxSequentialToolsInvocations` limits per agent role
- [ ] Enable Spring Boot virtual threads for concurrent I/O handling
- [ ] Centralize MCP endpoint configuration with environment variable overrides
- [ ] Implement transport-level timeouts and circuit breakers
- [ ] Enforce strict tool naming and parameter documentation standards
- [ ] Add observability hooks for tool call latency, success rates, and chain depth
- [ ] Create agent-specific tool namespaces to reduce prompt token overhead
- [ ] Test multi-tool chains with synthetic queries before production deployment

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| Single JVM, low latency, static tools | `@Tool` annotation | Zero network overhead, compile-time validation | Lowest compute cost |
| Cross-service, dynamic capabilities, multi-tenant | Dynamic MCP Provider | Decouples agent from service topology, enables runtime discovery | Moderate network cost, higher flexibility |
| High-throughput, strict rate limits, predictable workflows | Direct REST/gRPC clients | Bypasses LLM reasoning overhead, enables precise concurrency control | Lowest latency, highest development cost |
| Mixed environment (local + remote) | Hybrid provider + namespace filtering | Balances performance with distributed capability access | Optimized token usage, moderate infra cost |

### Configuration Template

```yaml
# application.yml
ai:
  mcp:
    services:
      order: ${MCP_ORDER_URL:http://localhost:3000/sse}
      payment: ${MCP_PAYMENT_URL:http://localhost:8091/sse}
      inventory: ${MCP_INVENTORY_URL:http://localhost:8092/sse}
      validation: ${MCP_VALIDATION_URL:http://localhost:8090/sse}
    transport:
      connect-timeout: 5s
      read-timeout: 10s
      log-requests: false
      log-responses: false
  agent:
    max-sequential-invocations: 5
    virtual-threads: true

spring:
  threads:
    virtual:
      enabled: true
  ai:
    chat:
      model:
        provider: openai
        options:
          model: gpt-4o
          temperature: 0.2

// Production-ready agent builder with observability hooks
@Bean
public ProductionAgentInterface secureAgent(
    McpToolProvider toolResolver,
    ChatModel languageModel,
    ToolInvocationMetrics metricsCollector
) {
    return AiServices.builder(ProductionAgentInterface.class)
        .chatModel(languageModel)
        .toolProvider(toolResolver)
        .maxSequentialToolsInvocations(5)
        .toolExecutionListener((call, result, duration) -> {
            metricsCollector.recordInvocation(call.name(), duration, result.success());
        })
        .build();
}

Quick Start Guide

1. Initialize MCP Servers: Deploy your backend services with MCP-compatible endpoints. Verify each exposes a /sse stream and responds to tools/list requests.
2. Configure Endpoints: Add service URLs to application.yml using environment variables for production and localhost defaults for development.
3. Register Tool Provider: Create a @Configuration class that builds McpToolProvider with your endpoint list and attaches it to AiServices.builder().
4. Enable Virtual Threads: Add spring.threads.virtual.enabled=true to your configuration to unlock concurrent tool execution.
5. Test Chain Execution: Submit a multi-step query (e.g., "Check stock for product X and calculate fraud risk for order Y"). Verify tool calls route correctly and complete within expected latency thresholds.

The dynamic MCP provider model transforms agents from static script runners into adaptive system operators. By enforcing execution boundaries, optimizing concurrency, and maintaining strict schema discipline, you can deploy distributed AI workflows that scale predictably under production load.