Difficulty

Intermediate

Read Time

8 min

Cómo construir un agente de IA con LangChain.js y NestJS: tutorial completo

By Codcompass Team·2026-07-04·8 min read

Architecting Production-Ready AI Agents: A Queue-Driven Approach with NestJS and LangChain

Current Situation Analysis

Integrating large language models into web applications introduces a fundamental architectural mismatch. Traditional HTTP frameworks are built around synchronous request-response cycles, expecting predictable latency and stateless interactions. LLMs, by contrast, are inherently asynchronous, stateful, and highly variable in execution time. When developers route AI calls directly through REST endpoints, they quickly encounter gateway timeouts, event loop blocking, and unmanageable scaling bottlenecks.

This problem is frequently overlooked because early-stage prototypes work flawlessly under low concurrency. A single developer testing gpt-4o via a direct HTTP call sees responses in 2-4 seconds and assumes the pattern will hold. In production, however, tool-calling agents introduce compounding latency. Each tool invocation requires a round-trip to the model, schema validation, and response parsing. With OpenAI's GPT-4o priced at approximately $5 per million tokens, a single multi-tool interaction typically consumes ~2,000 tokens. Multiply that by concurrent users, and you face three critical failures:

Timeout cascades: Load balancers terminate connections after 30-60 seconds, dropping in-flight AI jobs.
Resource exhaustion: Node.js event loops stall while waiting for external API responses, degrading all other endpoints.
Cost blindness: Without structured job tracking, token consumption becomes untraceable, leading to unexpected billing spikes.

The industry standard solution is to decouple the HTTP gateway from the AI execution layer using a message queue. This transforms unpredictable AI workloads into durable, retryable, and observable background tasks.

WOW Moment: Key Findings

Shifting from synchronous HTTP handling to a queue-driven architecture fundamentally changes how AI agents behave in production. The table below contrasts the two approaches across critical operational dimensions:

Approach	Request Latency	Horizontal Scalability	Failure Recovery	Token Cost Visibility
Direct HTTP	2-8s (blocks thread)	Limited by worker pool	Lost on crash	Hard to track per session
Queue-Driven	<200ms (async)	Infinite via worker scaling	Built-in retries & backoff	Granular per-job logging

Why this matters: Queue-driven execution converts AI interactions from a blocking liability into a managed workload. BullMQ provides exponential backoff, automatic retries, and job persistence via Redis. This means a temporary OpenAI rate limit or network hiccup no longer crashes the user experience. Instead, the job pauses, retries, and completes asynchronously. More importantly, every job carries metadata (session ID, token count, tool calls), enabling precise cost attribution and debugging. You stop guessing why an endpoint timed out and start tracing exactly which tool invocation consumed the budget.

Core Solution

The architecture follows a clean separation of concerns:

HTTP Client → NestJS Gateway → BullMQ Queue → LangChain Worker
                                                    ↓
                                           Zod-Validated Tools
                                                    ↓
                                           PostgreSQL (History) / Redis (Cache)

Step 1: Queue Registration & Module Setup

BullMQ requires a Redis connection and explicit queue configuration. We register the queue with production-safe defaults: exponential backoff for transient

failures, and automatic cleanup of completed jobs to prevent memory bloat.

// ai-orchestrator.module.ts
import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bullmq';
import { ConversationGateway } from './conversation.gateway';
import { TaskDispatcher } from './task.dispatcher';
import { AgentExecutorWorker } from './agent-executor.worker';

@Module({
  imports: [
    BullModule.registerQueue({
      name: 'ai-task-queue',
      defaultJobOptions: {
        attempts: 3,
        backoff: { type: 'exponential', delay: 2000 },
        removeOnComplete: 50,
        removeOnFail: false,
      },
    }),
  ],
  controllers: [ConversationGateway],
  providers: [TaskDispatcher, AgentExecutorWorker],
})
export class AiOrchestratorModule {}

Rationale: removeOnFail: false ensures failed jobs remain in Redis for manual inspection. Exponential backoff prevents thundering herd scenarios during API outages.

Step 2: HTTP Gateway & Task Dispatching

The controller never waits for the LLM. It validates the payload, enqueues the job, and returns a tracking identifier immediately.

// conversation.gateway.ts
import { Controller, Post, Body } from '@nestjs/common';
import { TaskDispatcher } from './task.dispatcher';

@Controller('conversations')
export class ConversationGateway {
  constructor(private readonly dispatcher: TaskDispatcher) {}

  @Post()
  async initiate(@Body() payload: { query: string; sessionId: string }) {
    const job = await this.dispatcher.schedule(payload);
    return { trackingId: job.id, status: 'processing' };
  }
}

// task.dispatcher.ts
import { Injectable } from '@nestjs/common';
import { Queue } from 'bullmq';
import { InjectQueue } from '@nestjs/bullmq';

@Injectable()
export class TaskDispatcher {
  constructor(@InjectQueue('ai-task-queue') private readonly queue: Queue) {}

  async schedule(data: { query: string; sessionId: string }) {
    return this.queue.add('execute-agent', data, {
      priority: 1,
      jobId: `session-${data.sessionId}-${Date.now()}`,
    });
  }
}

Rationale: Explicit jobId generation enables idempotency and client-side polling. Priority queuing allows premium sessions to bypass standard workloads if needed.

Step 3: LangChain Worker & Tool Orchestration

The worker consumes jobs, initializes the agent, and executes tool calls. We use Zod for strict schema validation and LangChain's native tool-calling agent for reliable function invocation.

// agent-executor.worker.ts
import { Processor, WorkerHost } from '@nestjs/bullmq';
import { Job } from 'bullmq';
import { ChatOpenAI } from '@langchain/openai';
import { AgentExecutor, createToolCallingAgent } from 'langchain/agents';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { tool } from '@langchain/core/tools';
import { z } from 'zod';

const retrieveKnowledge = tool(
  async (input) => {
    // Replace with actual DB/cache lookup
    return `Retrieved context for: ${input.searchTerm}`;
  },
  {
    name: 'retrieve_knowledge',
    description: 'Queries the internal knowledge base for relevant context',
    schema: z.object({
      searchTerm: z.string().min(3).describe('Keywords to search'),
    }),
  },
);

@Processor('ai-task-queue')
export class AgentExecutorWorker extends WorkerHost {
  private model = new ChatOpenAI({
    modelName: 'gpt-4o',
    temperature: 0.3,
    maxTokens: 1024,
  });

  async process(job: Job<{ query: string; sessionId: string }>) {
    const { query, sessionId } = job.data;

    const prompt = ChatPromptTemplate.fromMessages([
      ['system', 'You are a precise assistant. Use tools when available.'],
      ['placeholder', '{chat_history}'],
      ['human', '{input}'],
      ['placeholder', '{agent_scratchpad}'],
    ]);

    const agent = await createToolCallingAgent({
      llm: this.model,
      tools: [retrieveKnowledge],
      prompt,
    });

    const executor = new AgentExecutor({
      agent,
      tools: [retrieveKnowledge],
      maxIterations: 5,
      handleParsingErrors: true,
    });

    const result = await executor.invoke({
      input: query,
      chat_history: [], // Load from PostgreSQL based on sessionId
    });

    // Persist result to PostgreSQL
    // await this.historyRepository.save(sessionId, query, result.output);

    return result;
  }
}

Rationale:

maxIterations: 5 prevents infinite tool-calling loops.
handleParsingErrors: true gracefully recovers from malformed LLM outputs.
Zod enforces contract validation before the model ever sees the tool schema, reducing hallucination-induced crashes.
Chat history is intentionally left as a placeholder comment to emphasize that memory management belongs in a dedicated persistence layer, not the worker itself.

Pitfall Guide

1. Synchronous AI Blocking the Event Loop

Explanation: Awaiting LLM responses directly in a controller ties up Node.js threads. Under load, the entire application stalls. Fix: Always route AI workloads through a message queue. Return a job ID immediately and let workers handle execution asynchronously.

2. Unbounded Context Windows

Explanation: Passing full conversation history to every request causes token bloat, increased costs, and degraded model performance. GPT-4o's context window is 128k, but performance degrades significantly after ~10k tokens. Fix: Implement sliding window truncation. Keep only the last 8-12 exchanges in memory, and archive older messages to PostgreSQL. Use LangChain's BufferWindowMemory or custom truncation logic.

3. Ignoring Tool Schema Validation Failures

Explanation: LLMs occasionally generate malformed JSON or omit required fields. Without validation, the tool crashes and the agent enters an unrecoverable state. Fix: Wrap all tool invocations in Zod validation. Use safeParse to catch mismatches early and return structured error messages to the agent, allowing it to retry with corrected parameters.

4. Missing Rate Limit & Retry Strategy

Explanation: OpenAI enforces strict RPM/TPM limits. Burst traffic triggers 429 Too Many Requests errors, causing job failures. Fix: Configure BullMQ's rateLimit option and leverage LangChain's built-in retry mechanisms. Implement exponential backoff with jitter to distribute retry attempts evenly.

5. Hardcoding LLM Parameters

Explanation: Temperature, max tokens, and model versions change frequently. Hardcoding them requires redeployment for every adjustment. Fix: Externalize configuration to environment variables or a feature flag system. Inject parameters into the ChatOpenAI constructor dynamically.

6. Storing Raw AI Responses Without Metadata

Explanation: Saving only the final text output makes debugging impossible. You lose visibility into which tools were called, how many tokens were consumed, and where failures occurred. Fix: Persist structured metadata alongside responses. Log tokenUsage, toolCalls, latencyMs, and agentIterations to enable cost attribution and performance profiling.

7. Overcomplicating the Prompt Template

Explanation: Adding excessive system instructions or conflicting tool descriptions confuses the model, increasing latency and error rates. Fix: Keep prompts minimal and declarative. Use LangChain's ChatPromptTemplate placeholders strategically. Test prompt variations with A/B logging before promoting to production.

Production Bundle

Action Checklist

Queue Registration: Configure BullMQ with exponential backoff and job cleanup policies
Tool Schema Validation: Implement Zod schemas with min()/max() constraints for all agent tools
Context Management: Deploy sliding window truncation to cap chat history at 10 exchanges
Cost Tracking: Attach LangChain callback handlers to log token consumption per job
Error Boundaries: Wrap executor invocation in try/catch with structured error serialization
Health Monitoring: Expose queue depth, worker utilization, and average latency via Prometheus metrics
Idempotency: Generate deterministic job IDs based on session and timestamp to prevent duplicate processing

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Low traffic (<50 req/min)	Direct HTTP with connection pooling	Simpler architecture, lower operational overhead	Baseline token costs
Medium traffic (50-500 req/min)	BullMQ queue with 2-3 workers	Prevents timeout cascades, enables retry logic	+15% infrastructure, -30% failed requests
High traffic (>500 req/min)	Queue + auto-scaling workers + Redis cache	Handles burst loads, isolates AI latency from web tier	+40% infra, -80% latency variance
Strict compliance/audit	Queue + PostgreSQL persistence + callback logging	Full traceability, token accounting, and replay capability	+20% storage, enables precise billing

Configuration Template

# .env.production
REDIS_URL=redis://redis-cluster:6379
OPENAI_API_KEY=sk-xxxx
OPENAI_MODEL=gpt-4o
MAX_TOKENS=1024
TEMPERATURE=0.3
CONTEXT_WINDOW_SIZE=10
BULLMQ_QUEUE_NAME=ai-task-queue
BULLMQ_MAX_ATTEMPTS=3
BULLMQ_BACKOFF_DELAY=2000

// bullmq.config.ts
import { QueueOptions } from 'bullmq';

export const bullQueueConfig: QueueOptions = {
  connection: {
    url: process.env.REDIS_URL,
    maxRetriesPerRequest: null,
    enableReadyCheck: false,
  },
  defaultJobOptions: {
    attempts: Number(process.env.BULLMQ_MAX_ATTEMPTS) || 3,
    backoff: {
      type: 'exponential',
      delay: Number(process.env.BULLMQ_BACKOFF_DELAY) || 2000,
    },
    removeOnComplete: 50,
    removeOnFail: false,
  },
};

# Dockerfile
FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM node:22-alpine
WORKDIR /app
RUN addgroup -g 1001 appgroup && adduser -u 1001 -G appgroup -s /bin/sh -D appuser
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package*.json ./
USER appuser
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 CMD wget -qO- http://localhost:3000/health || exit 1
CMD ["node", "dist/main.js"]

Quick Start Guide

Initialize Project: Run nest new ai-agent-service and select npm as the package manager.
Install Dependencies: Execute npm install @nestjs/bullmq bullmq ioredis @langchain/core @langchain/openai @langchain/community zod.
Configure Environment: Create a .env file with Redis connection string and OpenAI API key. Update bullmq.config.ts to reference these variables.
Generate Modules: Run nest g module ai-orchestrator, nest g controller ai-orchestrator, and nest g service ai-orchestrator to scaffold the base structure.
Start Infrastructure: Launch Redis locally via docker run -d -p 6379:6379 redis:7-alpine. Run npm run start:dev to verify the queue accepts jobs and workers process them asynchronously.

🎉 Mid-Year Sale — Unlock Full Article

Base plan from just $4.99/mo or $49/yr

7-day free trial · Cancel anytime · 30-day money-back