MCP Client with LangChain4j

Architecting Multi-Service AI Agents with LangChain4j and MCP

Current Situation Analysis

Modern AI agents rarely operate in isolation. As enterprise applications adopt agentic workflows, the demand for agents to interact with distributed backend services has exploded. The traditional approach involves hardcoding tool definitions directly into the agent's JVM. This creates a monolithic coupling where every new service capability requires a redeployment of the agent, tight versioning dependencies, and bloated context windows filled with irrelevant tool schemas.

The industry pain point is scalability and maintainability. When an agent needs access to capabilities across Order, Payment, Inventory, and Validation services, static tool registration becomes unmanageable. Developers face a dilemma: either duplicate service logic into the agent or build complex, brittle RPC wrappers.

The Model Context Protocol (MCP) resolves this by decoupling tool definition from tool execution. However, integrating MCP into Java-based agent frameworks introduces new challenges around transport configuration, dynamic schema resolution, and runtime orchestration. LangChain4j's McpToolProvider addresses these by treating MCP servers as dynamic tool sources.

Data from production implementations highlights the impact of this architecture. In a multi-service scenario involving 12+ tools across 4 distinct microservices, static registration resulted in high coupling and sequential execution bottlenecks. Switching to an MCP-driven provider reduced tool discovery latency and enabled runtime parallelization. Crucially, enabling virtual threads in this architecture reduced end-to-end latency for multi-tool chains from approximately 8 seconds to 3 seconds, a 62% improvement driven by non-blocking I/O during tool execution.

WOW Moment: Key Findings

The shift from static tool registration to dynamic MCP provisioning fundamentally changes agent architecture. The following comparison illustrates the operational differences between a monolithic tool approach and an MCP-integrated agent.

Approach	Coupling	Tool Discovery	Parallelism Support	Latency (5-Tool Chain)	Maintenance Overhead
Static Registration	High (Compile-time)	Manual/Code Changes	Manual Implementation	~8.0s	High (Redeploy agent for new tools)
MCP Dynamic Provider	Low (Runtime)	Automatic via tools/list	Automatic (Virtual Threads)	~3.0s	Low (Services update independently)

Why this matters: The MCP approach transforms the agent from a rigid orchestrator into a flexible consumer. The agent no longer needs to know the implementation details or network locations of the tools it uses. It interacts with a unified interface, while the McpToolProvider handles routing, serialization, and error translation. This enables independent scaling of services and rapid iteration on tool capabilities without touching the agent codebase.

Core Solution

Implementing an MCP-integrated agent requires three distinct phases: transport configuration, provider assembly, and agent wiring. The goal is to create a resilient bridge that dynamically resolves tools from remote services while maintaining strict safety controls.

Step 1: Transport and Client Configuration

Each MCP server exposes tools via a transport layer, typically Server-Sent Events (SSE) over HTTP. You must configure individual clients for each service endpoint. This abstraction allows the agent to treat heterogeneous services uniformly.

public class McpClientFactory {

    public static McpClient createClient(String endpointUrl) {
        HttpMcpTransport transport = HttpMcpTransport.builder()
                .sseUrl(endpointUrl)
                .logRequests(true)
                .logResponses(true)
                .build();

        return DefaultMcpClient.builder()
                .transport(transport)
                .build();
    }
}

Rationale: Logging requests and responses is critical during development and debugging. In production, you may disable verbose logging but retain structured tracing. The factory pattern centralizes client creation, ensuring consistent transport settings across all service connections.

Step 2: Assembling the Tool Provider

The McpToolProvider aggregates multiple clients into a single source of truth. It queries each server for available tools during initialization and caches the schemas.

@Configuration
public class AgentToolingConfig {

    @Value("${mcp.services.order}")
    private String orderServiceUrl;

    @Value("${mcp.services.payment}")
    private String paymentServiceUrl;

    @Value("${mcp.services.inventory}")
    private String inventoryServiceUrl;

    @Bean
    public McpToolProvider dynamicToolProvider() {
        List<McpClient> clients = List.of(
                McpClientFactory.createClient(orderServiceUrl),
                McpClientFactory.createClient(paymentServiceUrl),
                McpClientFactory.createClient(inventoryServiceUrl)
        );

        return McpToolProvider.builder()
                .mcpClients(clients)
                .build();
    }
}

Rationale: Centralizing the provider definition in a configuration class aligns with Spring Boot best practices. Environment variables drive the URLs, allowing seamless promotion across environments. The provider handles the tools/list protocol calls automatically, shielding the agent from service discovery logic.

Step 3: Wiring the Agent

With the provider ready, you inject it into the agent builder. This replaces static tool lists with dynamic resolution.

@Service
public class AgentOrchestrator {

    private final ChatModel chatModel;
    private final McpToolProvider toolProvider;

    public AgentOrchestrator(ChatModel chatModel, McpToolProvider toolProvider) {
        this.chatModel = chatModel;
        this.toolProvider = toolProvider;
    }

    public BusinessAnalystAgent buildAnalyst() {
        return AiServices.builder(BusinessAnalystAgent.class)
                .chatModel(chatModel)
                .toolProvider(toolProvider)
                .maxSequentialToolsInvocations(6)
                .build();
    }
}

Rationale: maxSequentialToolsInvocations is a mandatory safety guard. Without it, an LLM could enter an infinite loop of tool calls, consuming resources and tokens. The limit should be tuned based on the complexity of expected workflows. For complex multi-service queries, a limit of 5-6 is typical; simpler agents may require only 2-3.

Architecture Decisions

• SSE Transport: SSE is preferred for real-time tool streaming and reliable connection management compared to raw HTTP polling.
• Virtual Threads: Tool execution involves network I/O. Enabling virtual threads allows the framework to suspend waiting threads and resume others, drastically improving throughput for multi-tool chains.
• Provider vs. Static Tools: Use McpToolProvider for cross-service capabilities. Reserve @Tool annotations for local, same-JVM utilities that do not require network calls.

Pitfall Guide

1. The Runaway Agent Loop

Explanation: LLMs can occasionally misinterpret tool outputs or enter recursive reasoning patterns, calling tools indefinitely. Fix: Always configure maxSequentialToolsInvocations. Monitor tool call counts in telemetry to detect anomalies. Set the limit based on the maximum expected chain length plus a small buffer.

2. Thread Starvation on I/O

Explanation: Each MCP tool call is an HTTP request. In a thread-per-request model, sequential tool calls block threads, causing latency multiplication. Fix: Enable virtual threads in your runtime configuration. This allows the agent to handle multiple independent tool calls concurrently without exhausting thread pools.

spring:
  threads:
    virtual:
      enabled: true

3. Initialization Blocking

Explanation: If an MCP server is down during agent startup, the McpToolProvider may fail to fetch the tool list, preventing the agent from initializing. Fix: Implement health checks or lazy initialization strategies. In critical systems, consider a fallback mode where the agent starts with a subset of tools and retries failed connections asynchronously.

4. Context Window Bloat

Explanation: Aggregating tools from multiple services can result in a large number of tool schemas, consuming valuable context window space. Fix: Optimize tool descriptions in the MCP servers. Use concise, high-signal descriptions. Group related tools logically. Monitor token usage to ensure tool definitions do not crowd out conversation history.

5. Silent Tool Failures

Explanation: Network timeouts or service errors may result in empty or malformed responses, causing the LLM to hallucinate or proceed with incorrect data. Fix: Ensure MCP servers return structured error messages. The McpToolProvider translates these into tool responses that the LLM can interpret. Implement retry logic for transient errors at the transport layer.

6. Schema Drift

Explanation: If an MCP server updates a tool signature without updating the description, the LLM may generate invalid arguments. Fix: Maintain strict versioning of tool schemas. Use automated tests to validate that tool descriptions match implementation signatures. Implement schema validation in the MCP servers to reject malformed calls early.

Production Bundle

Action Checklist

• Enable Virtual Threads: Configure spring.threads.virtual.enabled=true to maximize tool execution throughput.
• Set Invocation Limits: Define maxSequentialToolsInvocations based on workflow complexity to prevent infinite loops.
• Configure Endpoints via Env Vars: Externalize MCP server URLs to support environment-specific deployments.
• Implement Error Handling: Wrap agent execution in try-catch blocks to prevent application crashes on tool failures.
• Monitor Tool Usage: Instrument telemetry to track tool call frequency, latency, and error rates.
• Optimize Tool Descriptions: Review and refine tool schemas to minimize context window consumption.
• Test Multi-Tool Chains: Validate complex queries that require chaining tools across multiple services.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Same-JVM Utility	@Tool Annotation	Zero network overhead, direct method invocation.	Low latency, high coupling.
Cross-Service Capability	McpToolProvider	Decouples agent from service implementation; enables independent scaling.	Network overhead, loose coupling.
High Concurrency	Virtual Threads + MCP	Maximizes throughput for I/O-bound tool calls.	Minimal CPU overhead, improved latency.
Strict Latency SLA	Local Caching + MCP	Cache frequent tool results to reduce network calls.	Memory cost, reduced latency.

Configuration Template

# application.yml
mcp:
  services:
    order: ${ORDER_MCP_URL:http://localhost:3000/sse}
    payment: ${PAYMENT_MCP_URL:http://localhost:8091/sse}
    inventory: ${INVENTORY_MCP_URL:http://localhost:8092/sse}
    validation: ${VALIDATION_MCP_URL:http://localhost:8090/sse}

spring:
  threads:
    virtual:
      enabled: true

// AgentConfig.java
@Configuration
public class AgentConfig {

    @Bean
    public McpToolProvider mcpToolProvider(
            @Value("${mcp.services.order}") String orderUrl,
            @Value("${mcp.services.payment}") String paymentUrl,
            @Value("${mcp.services.inventory}") String inventoryUrl) {
        
        List<McpClient> clients = List.of(
                McpClientFactory.createClient(orderUrl),
                McpClientFactory.createClient(paymentUrl),
                McpClientFactory.createClient(inventoryUrl)
        );

        return McpToolProvider.builder()
                .mcpClients(clients)
                .build();
    }
}

Quick Start Guide

1. Add Dependencies: Include LangChain4j MCP and transport dependencies in your build configuration.
2. Configure Endpoints: Define MCP server URLs in application.yml using environment variables.
3. Build Provider: Create a McpToolProvider bean that aggregates clients for each service.
4. Wire Agent: Inject the provider into AiServices.builder and set maxSequentialToolsInvocations.
5. Enable Virtual Threads: Set spring.threads.virtual.enabled=true to optimize I/O performance.
6. Test: Execute a multi-tool query to verify dynamic tool resolution and cross-service chaining.