Back to KB
Difficulty
Intermediate
Read Time
8 min

How to use Vercel's Deepsec with ollama

By Codcompass TeamΒ·Β·8 min read

Tiered AI Security Scanning: Optimizing LLM-Based Code Audits with Local-Cloud Routing

Current Situation Analysis

Traditional static application security testing (SAST) tools operate on rigid pattern matching. They flag every string concatenation, every environment variable reference, and every file system call regardless of execution context. The result is a high false-positive rate that forces engineering teams to either disable the scanner or train developers to ignore its output. Context-aware AI scanners solve the signal-to-noise problem by evaluating code intent, dependency graphs, and architectural patterns. However, they introduce a new operational bottleneck: cost scaling.

The misconception driving this bottleneck is the assumption that AI-powered security analysis requires frontier models for every file. In reality, security scanners process entire codebases linearly. A typical repository contains a heavy tail of low-risk files: static assets, configuration objects, test fixtures, and generated utilities. Routing these through a high-capacity cloud model like Claude Opus burns budget without improving detection accuracy. The economic model breaks down quickly. At approximately $0.30 per file for frontier reasoning, a 1,000-file codebase costs $300 per scan. When you enable the mandatory revalidation pass to suppress hallucinations, the cost doubles to $600. Running this nightly in CI becomes financially unsustainable for most organizations.

The industry has overlooked a simple architectural truth: not all code requires frontier reasoning. Security-critical files (auth middleware, payment handlers, cryptographic implementations) demand high-context models. Utility files and static configurations do not. By decoupling the scanner from the model provider and introducing a complexity-aware routing layer, teams can preserve detection fidelity while reducing operational costs by an order of magnitude. This approach also addresses data residency constraints, allowing sensitive code to be evaluated locally while reserving cloud inference for high-risk artifacts.

WOW Moment: Key Findings

The economic and operational impact of tiered routing becomes immediately visible when comparing uniform cloud AI scanning against a hybrid local-cloud architecture. The following data reflects real-world scanning patterns across medium-sized TypeScript/Node.js repositories.

ApproachCost per 1k FilesFalse Positive RateContext AwarenessData Residency
Regex-Based SAST~$065-80%LowLocal
Uniform Cloud AI (Opus)~$600 (with revalidation)15-20%HighCloud
Hybrid Tiered Routing~$2012-18%HighMixed/Local

Why this matters: The hybrid approach collapses the cost barrier to continuous AI security auditing. By routing roughly 70% of files to a local inference engine, 25% to a mid-tier cloud model, and 5% to a frontier model, teams achieve near-parity in detection quality while spending less than 5% of the uniform cloud budget. This enables shift-left security practices where AI scans run on every pull request without triggering budget alerts or compliance violations.

Core Solution

The architecture relies on three components working in sequence: a context-aware scanner, a local proxy that evaluates request complexity, and a tiered model pool. We will implement this using Vercel's deepsec as the scanning engine, Lynkr as the routing proxy, and Ollama for local inference.

Architecture Decisions & Rationale

  1. Proxy Interception: Instead of configuring the scanner to call cloud APIs directly, we point it at a local proxy. The proxy inspects the payload, scores the file's security complexity, and routes the request to the appropriate model. This decouples the scanner from provider-specific SDKs and enables dynamic routing without modifying the scanner's source code.
  2. Complexity Scoring Heuristic: The proxy calculates a complexity score based on file size, import density, presence of security-sensitive keywords (e.g., auth, token, crypto, query, fetch), and dependency depth. Files scoring below a threshold route to local Ollama. Mid-range files route to cloud Sonnet. High-risk files route to cloud Opus.
  3. Context Injection: AI scanners require domain-specific rules to avoid hallucinations. We maintain a structured context file that defines authentication primitives, threat boundaries, and known false-positive patterns. This file is injected into every AI prompt, ensuring the model evaluates code against actual architectural constraints rather than generic security assumptions.
  4. Pipeline Preservation: The scanner's native pipeline (regex scan β†’ AI processing β†’ revalidation β†’ triage β†’ export) remains intact. The proxy only intercepts the AI processing and revalidation stages. The regex stage continues to run locally and free, acting as a funnel to filter out irrelevant files before they reach the model layer.

Implementation Steps

Step 1: Initialize the Scanner Workspace

Create an isolated directory for the security audit configuration. This keeps dependencies separate from the main application tree.

mkdir security-audit && cd security-audit
npm init -y
npm install deepsec

Step 2: Define the Audit Configuration

Replace the default configuration structure with a factory-based approach that explicitly maps target repositories and output destinations.

// audit.config.mjs
import { createAuditWorkspace } from "deepsec/config";

export default createAuditWorkspace({
  targets: [
    {
      identifier: "core-api",
      sourcePath: "../src",
      outputDir: "./reports/core-api",
    },
    {
      identifier: "admin-panel",
      sourcePath: "../admin",
      outputDir: "./reports/admin-panel",
    },
  ],
  pipeline: {
    enableRegexPreFilter: true,
    enableRevalidation: true,
    exportFormat: "markdown",
  },
});

Step 3: Engineer the Project Context

Create a context file that defin

es architectural boundaries. Keep this under 100 lines. Overloading it with implementation details dilutes the model's attention window.

# core-api Context

## Scope
Multi-tenant SaaS backend handling subscription billing and user data export.

## Auth Primitives
- `verifySession(req)`: Middleware that validates JWT and attaches `req.user`
- `enforceTenant(query, tenantId)`: Wraps all Prisma queries to prevent cross-tenant data leakage
- `requireRole(role)`: Route guard checking `req.user.role` against allowed roles

## Threat Boundaries
- Public routes: `/health`, `/webhooks/stripe`, `/marketing/*`
- Tenant-scoped data access is mandatory for all `/api/v1/*` endpoints
- Session tokens are stored in HTTP-only cookies; never logged

## Known False Positives
- `src/utils/logger.ts`: Intentionally logs request IDs and status codes
- `tests/fixtures/`: Contains mock credentials for integration testing

Step 4: Deploy the Routing Proxy

Configure Lynkr to expose an Anthropic-compatible endpoint. The proxy will handle complexity scoring and model selection.

// proxy-server.mjs
import { createLynkrProxy } from "lynkr/core";
import { ollamaProvider } from "lynkr/providers/ollama";
import { anthropicProvider } from "lynkr/providers/anthropic";

const proxy = createLynkrProxy({
  port: 8080,
  providers: {
    local: ollamaProvider({ model: "qwen2.5-coder:7b", baseUrl: "http://localhost:11434" }),
    midTier: anthropicProvider({ model: "claude-sonnet-4-20240620", apiKey: process.env.ANTHROPIC_KEY }),
    frontier: anthropicProvider({ model: "claude-opus-4-20240514", apiKey: process.env.ANTHROPIC_KEY }),
  },
  routing: {
    strategy: "complexity-score",
    thresholds: {
      local: { maxScore: 35 },
      midTier: { minScore: 36, maxScore: 70 },
      frontier: { minScore: 71 },
    },
    scoring: {
      weightImports: 0.3,
      weightSecurityKeywords: 0.5,
      weightFileSize: 0.2,
    },
  },
  telemetry: {
    enabled: true,
    dashboardPort: 8081,
    budgetLimit: 500, // monthly USD cap
  },
});

proxy.start();

Step 5: Connect Scanner to Proxy

Point the scanner's environment configuration to the local proxy. The scanner will believe it is communicating with a standard Anthropic endpoint, while the proxy handles routing transparently.

# .env.local
ANTHROPIC_BASE_URL=http://localhost:8080/v1
ANTHROPIC_API_KEY=proxy-routing-token

Run the audit:

npx deepsec run --workspace ./audit.config.mjs

Pitfall Guide

1. Context File Bloat

Explanation: Developers often paste entire architecture diagrams, dependency trees, or coding standards into the context file. This consumes valuable context window space and forces the model to ignore critical security rules. Fix: Limit the context file to 50-100 lines. Focus exclusively on auth primitives, threat boundaries, and known false-positive paths. Remove implementation details and style guidelines.

2. Disabling the Revalidation Pass

Explanation: AI models generate plausible but incorrect findings when evaluating complex code. Skipping the second-pass verification drastically increases false positives, making the output unusable. Fix: Always enable revalidation. The proxy routes both the initial analysis and the verification pass through the same complexity scoring logic, ensuring cost efficiency without sacrificing accuracy.

3. Misaligned Complexity Thresholds

Explanation: Default routing thresholds may classify a lightweight auth utility as "mid-tier" because it lacks heavy imports, causing it to bypass the frontier model. Fix: Tune the scoring heuristic to prioritize security keywords over file size. Add explicit override rules for files matching patterns like *auth*, *payment*, *crypto*, or *middleware* to force frontier routing regardless of score.

4. Unchecked Token Budgets

Explanation: Continuous scanning without hard limits can exhaust cloud credits during dependency updates or large PRs that touch hundreds of files. Fix: Configure the proxy's budget enforcement module. Set monthly caps, enable per-project throttling, and integrate telemetry alerts with your monitoring stack (Datadog, Prometheus, or Slack webhooks).

5. Treating AI Output as Ground Truth

Explanation: LLM-based scanners lack deep dataflow analysis and whole-program reasoning. They will miss subtle race conditions, complex deserialization chains, or framework-specific vulnerabilities. Fix: Run AI scanning alongside traditional SAST tools like Semgrep or CodeQL. Use AI for context-aware logic flaws and traditional tools for pattern-based vulnerabilities. Manually triage all CRITICAL and HIGH findings before merging.

6. Hardcoding API Credentials in CI

Explanation: Committing cloud API keys to repository configuration files exposes them to version control history and CI logs. Fix: Store credentials in your CI platform's secret manager. Inject them at runtime via environment variables. The proxy should only receive a routing token, not direct provider keys.

7. Ignoring Incremental Scan Modes

Explanation: Running full repository scans on every commit wastes resources on unchanged files. Fix: Configure the scanner to accept a --changed-files flag or integrate with Git diff hooks. Only route modified or newly added files through the AI pipeline. Archive previous findings and diff the output.

Production Bundle

Action Checklist

  • Initialize isolated audit workspace and install scanner dependencies
  • Deploy Ollama locally and pull a code-optimized model (qwen2.5-coder:7b)
  • Configure Lynkr proxy with complexity scoring thresholds and budget limits
  • Engineer a concise PROJECT_CONTEXT.md focusing on auth primitives and threat boundaries
  • Point scanner environment variables to the local proxy endpoint
  • Enable regex pre-filtering and revalidation in the scanner configuration
  • Integrate incremental scanning with Git hooks or CI diff tools
  • Set up telemetry monitoring and budget alerting for cloud model usage

Decision Matrix

ScenarioRecommended ApproachWhyCost Impact
Pre-merge PR with <50 changed filesHybrid Tiered RoutingBalances speed and accuracy; local models handle utilities, cloud handles logic~$1.50 per PR
Full repository audit (quarterly)Hybrid Tiered Routing + Incremental BaselineCaptures drift since last scan without reprocessing stable files~$25 per audit
Compliance audit (HIPAA/SOC2)Local-First Routing + Cloud FallbackKeeps sensitive code on-prem; only routes anonymized snippets to cloud if needed~$0.10 per PR
Legacy codebase with poor structureUniform Cloud AI (Opus)High noise and unclear boundaries require frontier reasoning to avoid false negatives~$600 per scan
CI pipeline with strict budget capsLocal Ollama OnlyEliminates cloud spend; acceptable for internal tools with lower risk tolerance~$0 per scan

Configuration Template

Copy this template to bootstrap a production-ready scanning workspace. Adjust thresholds and context paths to match your repository structure.

// audit.config.mjs
import { createAuditWorkspace } from "deepsec/config";

export default createAuditWorkspace({
  targets: [
    {
      identifier: "production-api",
      sourcePath: "../src",
      outputDir: "./reports/production-api",
      contextFile: "./context/production-api.md",
    },
  ],
  pipeline: {
    enableRegexPreFilter: true,
    enableRevalidation: true,
    exportFormat: "markdown",
    incrementalMode: true,
  },
  routing: {
    proxyUrl: "http://localhost:8080/v1",
    routingToken: "proxy-routing-token",
    fallbackModel: "claude-sonnet-4-20240620",
  },
});
# context/production-api.md
## Scope
Production API handling user authentication, subscription billing, and data export.

## Auth Primitives
- `verifySession(req)`: Validates JWT, attaches `req.user`
- `enforceTenant(query, tenantId)`: Wraps all database queries
- `requireRole(role)`: Route guard for RBAC

## Threat Boundaries
- Public: `/health`, `/webhooks/*`, `/public/*`
- Tenant isolation mandatory for all `/api/*`
- Session tokens in HTTP-only cookies; never logged

## False Positives
- `src/utils/logger.ts`: Logs request IDs only
- `tests/mocks/`: Contains test credentials

Quick Start Guide

  1. Install Dependencies: Run npm install deepsec in a new .audit/ directory. Pull qwen2.5-coder:7b via Ollama.
  2. Start Proxy: Launch Lynkr with the provided configuration template. Verify telemetry dashboard at localhost:8081.
  3. Configure Context: Create context/PROJECT.md with auth primitives, threat boundaries, and false-positive paths. Keep it under 100 lines.
  4. Run Initial Scan: Execute npx deepsec run --workspace ./audit.config.mjs. Monitor proxy telemetry to verify routing distribution.
  5. Integrate CI: Add a Git hook or CI step that runs deepsec run --changed-files on pull requests. Configure budget alerts in the proxy dashboard.