I built agmsg so Claude Code and Codex could stop using me as a copy-paste relay
Local SQLite IPC for Multi-Agent CLI Workflows
Current Situation Analysis
Modern development workflows increasingly rely on orchestrating multiple Large Language Model (LLM) agents simultaneously. A common pattern involves using a high-throughput agent (e.g., Claude Code) for iterative coding and a reasoning-heavy agent (e.g., Codex or Opus-class models) for architectural review or complex debugging. While this hybrid approach leverages the distinct strengths of each model, it introduces a significant operational friction: the Human Bridge Anti-Pattern.
Developers frequently become a manual relay between agents. Context, code snippets, and feedback must be copied from one agent's interface and pasted into another. This process is not merely tedious; it degrades engineering velocity and introduces error vectors.
- Cognitive Fragmentation: Every manual transfer requires the developer to parse output, select relevant context, and reformat input for the receiving agent. This breaks flow state and increases context-switching overhead.
- Error Propagation: Manual copy-paste operations are prone to truncation, omission of critical error logs, or version mismatches where the receiving agent acts on stale context.
- Scalability Limits: As the number of agents increases, the relay complexity grows factorially. Managing three agents manually becomes unmanageable within minutes.
This problem is often overlooked because most agent tools are designed as isolated silos. Built-in "sub-agent" features in some CLI tools typically support short-lived, hierarchical task delegation rather than persistent, peer-to-peer communication. Consequently, engineers accept the manual relay as the cost of using multiple models, despite the availability of simpler architectural solutions.
WOW Moment: Key Findings
The breakthrough in multi-agent orchestration lies in decoupling communication from the agents themselves. By introducing a lightweight, shared state layer, agents can exchange messages asynchronously without human intervention.
The following comparison demonstrates the impact of replacing manual relays with a local SQLite-based Inter-Process Communication (IPC) mechanism.
| Approach | Latency | Concurrency | Setup Complexity | Error Risk | Persistence |
|---|---|---|---|---|---|
| Manual Relay | High (Human-dependent) | None | None | High | None |
| File-Based IPC | Medium | Low (Locking issues) | Low | Medium | Yes |
| SQLite IPC (WAL) | Low (Real-time) | High | Minimal | Low | Yes |
| Networked Bus (Redis) | Low | High | High | Low | Yes |
Key Insight: SQLite in Write-Ahead Logging (WAL) mode provides "multiple readers, single writer" semantics with zero configuration. This allows multiple agents to poll or monitor a shared database concurrently without file corruption or the need for a daemon process. The result is a robust message bus that requires only bash and sqlite3, eliminating infrastructure overhead while enabling real-time, bidirectional communication between heterogeneous agents.
Core Solution
The architecture centers on a shared SQLite database acting as the message bus. Each agent interacts with the database through a standardized interface, abstracting the underlying delivery mechanism. This design supports heterogeneous agents (e.g., Claude Code, Codex, Gemini CLI) by normalizing communication regardless of their native hook or configuration models.
Architecture Decisions
- SQLite over Text Files: Text files fail under concurrent access. Two agents writing to the same file risk corruption or lost updates. SQLite handles concurrency natively.
- WAL Mode: Enabling WAL mode allows readers to proceed without blocking writers and vice versa. This is critical for agents that may read messages while another is writing.
- Zero-Daemon Design: Running a separate process for the message bus adds deployment complexity and failure points. SQLite operates in-process, requiring no background services.
- Minimal Dependencies: The solution relies only on
bashandsqlite3, ensuring portability across development environments without language runtime requirements.
Implementation: llm-ipc
The following TypeScript/Bash hybrid implementation demonstrates the core logic. This example uses a schema optimized for message routing and status tracking.
1. Database Schema and Initialization
// llm-ipc.ts
import { execSync } from 'child_process';
const DB_PATH = process.env.LLM_IPC_DB || './.agent_bus.db';
export function initBus(): void {
const schema = `
PRAGMA journal_mode=WAL;
PRAGMA busy_timeout=5000;
CREATE TABLE IF NOT EXISTS agents (
name TEXT PRIMARY KEY,
team TEXT NOT NULL,
last_seen DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sender TEXT NOT NULL,
recipient TEXT NOT NULL,
payload TEXT NOT NULL,
type TEXT DEFAULT 'text',
status TEXT DEFAULT 'pending',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_recipient_status
ON messages(recipient, status, created_at);
`;
execSync(`sqlite3 "${DB_PATH}" "${schema}"`);
console.log(`[llm-ipc] Bus initialized at ${DB_PATH}`);
}
Rationale:
PRAGMA busy_timeout=5000: Prevents immediate failures if the database is locked briefly.agentstable: Tracks active participants and teams for isolation.statusfield: Enables atomic fetch-and-update patterns, preventing duplicate message processing.- Index on
(recipient, status, created_at): Optimizes queries for retrieving pending messages for a specific agent.
2. Message Sending
export function sendMessage(recipient: string, payload: string, sender: string): void {
const escapedPayload = payload.replace(/'/g, "''");
const query = `
INSERT INTO messages (sender, recipient, payload)
VALUES ('${sender}', '${recipient}', '${escapedPayload}');
`;
execSync(`sqlite3 "${DB_PATH}" "${query}"`);
}
3. Atomic Message Reception
export function receiveMessage(agentName: string): string | null {
// Use a transaction to atomically select and mark the message
const query = `
BEGIN IMMEDIATE;
SELECT payload FROM messages
WHERE recipient='${agentName}' AND status='pending'
ORDER BY created_at ASC LIMIT 1;
UPDATE messages SET status='read'
WHERE recipient='${agentName}' AND status='pending'
ORDER BY created_at ASC LIMIT 1;
COMMIT;
`;
const result = execSync(`sqlite3 "${DB_PATH}" "${query}"`).toString().trim();
return result || null;
}
Rationale:
- The
BEGIN IMMEDIATEandCOMMITblock ensures that the selection and status update are atomic. This prevents race conditions where two agents might attempt to read the same message. - Ordering by
created_atensures FIFO processing.
4. Delivery Modes
To support heterogeneous agents, the system abstracts delivery into three modes:
- Monitor Mode: For agents with real-time monitoring capabilities (e.g., Claude Code's Monitor tool), the system subscribes to the database. When a new message is detected, the agent is interrupted immediately.
- Hook Mode: For agents without monitoring, the system integrates with turn-based hooks (e.g.,
Stophooks). The agent checks for messages between turns. - Manual Mode: The agent queries the inbox on demand via a command interface.
export function startMonitor(agentName: string, callback: (msg: string) => void): void {
// Polling loop with exponential backoff or blocking read
// Implementation depends on agent capabilities
const poll = async () => {
const msg = receiveMessage(agentName);
if (msg) callback(msg);
setTimeout(poll, 1000); // Adjust interval based on latency requirements
};
poll();
}
Pitfall Guide
Implementing multi-agent IPC introduces specific challenges. The following pitfalls and fixes are derived from production experience with local agent orchestration.
1. Text File Race Conditions
Explanation: Using plain text files for message passing leads to corruption when multiple agents write simultaneously. File locking is inconsistent across platforms and agents. Fix: Use SQLite with WAL mode. It provides ACID compliance and handles concurrent access safely without external locking mechanisms.
2. Ignoring WAL Mode
Explanation: Default SQLite journaling can cause "database is locked" errors under concurrent access, especially when agents poll frequently.
Fix: Always execute PRAGMA journal_mode=WAL; during initialization. This allows concurrent readers and a single writer, significantly improving throughput.
3. Unbounded Message Growth
Explanation: Over time, the message table grows indefinitely, increasing database size and query latency.
Fix: Implement a cleanup routine. Periodically delete messages older than a threshold or with status='read'.
DELETE FROM messages WHERE status='read' AND created_at < datetime('now', '-24 hours');
4. Hook Fragmentation Across Agents
Explanation: Different agents have varying hook models. Codex may lack the monitoring capabilities of Claude Code, leading to inconsistent delivery. Fix: Abstract delivery modes. Detect agent capabilities at startup and configure the appropriate delivery mechanism (Monitor vs. Hook vs. Manual). Ensure the interface remains consistent regardless of the underlying mode.
5. Blocking the Agent Loop
Explanation: A naive polling loop can consume excessive CPU or block the agent's main thread, degrading performance. Fix: Use efficient polling with backoff or leverage the agent's native asynchronous capabilities. For monitor mode, use blocking reads where supported to minimize CPU usage.
6. Security Assumptions
Explanation: Local IPC assumes a trusted environment. However, message payloads may contain sensitive code or credentials. Fix: Restrict database file permissions. Ensure the database is stored in a secure workspace directory and is not committed to version control. Consider encrypting sensitive payloads if the environment is shared.
Production Bundle
Action Checklist
- Initialize Database: Run the initialization script to create the SQLite database with WAL mode and proper schema.
- Configure Agents: Set environment variables (
LLM_IPC_DB,LLM_IPC_AGENT,LLM_IPC_TEAM) for each agent instance. - Install Skills: Deploy the
llm-ipcskill to each agent, ensuring compatibility with their respective hook models. - Verify Delivery: Test message passing between agents using manual mode to confirm routing and status updates.
- Enable Monitor: Activate monitor mode for agents that support it to achieve real-time communication.
- Add Cleanup: Schedule a periodic cleanup job to prune read messages and manage database size.
- Secure Workspace: Ensure the database file is excluded from version control and has restrictive file permissions.
Decision Matrix
| Scenario | Recommended Approach | Why | Cost Impact |
|---|---|---|---|
| Solo Developer, Local | SQLite IPC | Zero infrastructure, fast setup, reliable concurrency | Free |
| Team, Remote Agents | Redis/gRPC | Network support, scalability, centralized management | Infra cost |
| Quick Prototype | Text Files | Simple implementation, no dependencies | High risk of errors |
| High-Volume Production | Message Queue (Kafka) | Durability, ordering guarantees, monitoring | Complex setup |
Configuration Template
# llm-ipc.config.yaml
db_path: ./workspace/.agent_bus.db
team: dev_squad_alpha
delivery_mode: auto # auto, monitor, hook, manual
cleanup:
enabled: true
retention_hours: 24
max_size_mb: 50
security:
file_permissions: "0600"
encrypt_payloads: false # Enable if sensitive data is transmitted
Quick Start Guide
- Install Dependencies: Ensure
sqlite3is available in your environment.sudo apt-get install sqlite3 # Debian/Ubuntu brew install sqlite3 # macOS - Initialize the Bus: Run the initialization command to set up the database.
llm-ipc init --db ./workspace/.agent_bus.db - Configure Agent A: Set environment variables and start the agent.
export LLM_IPC_DB=./workspace/.agent_bus.db export LLM_IPC_AGENT=claude_driver export LLM_IPC_TEAM=dev_squad_alpha claude-code - Configure Agent B: Repeat for the second agent with a unique name.
export LLM_IPC_AGENT=codex_reviewer codex - Test Communication: Send a test message from Agent A to Agent B.
Agent B should receive the message via the configured delivery mode.llm-ipc send --to codex_reviewer --msg "Review complete. Ready for next task."
By implementing a local SQLite IPC layer, development teams can eliminate the human bridge anti-pattern, enabling seamless, real-time collaboration between AI agents. This architecture provides a robust, scalable foundation for multi-agent workflows while maintaining simplicity and security.
Mid-Year Sale β Unlock Full Article
Base plan from just $4.99/mo or $49/yr
Sign in to read the full article and unlock all tutorials.
Sign In / Register β Start Free Trial7-day free trial Β· Cancel anytime Β· 30-day money-back
