Running Local AI (Self-hosted) Coding Assistants in VS Code with Ollama and GitHub Copilot

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Enable and start the service
sudo systemctl enable --now ollama

sudo mkdir -p /etc/systemd/system/ollama.service.d
cat <<EOF | sudo tee /etc/systemd/system/ollama.service.d/override.conf
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
EOF

sudo systemctl daemon-reload
sudo systemctl restart ollama

els.

Pull and verify the model:

# Pull the recommended coding model
ollama pull qwen2.5-coder:14b

# Verify the model is available
ollama list | grep qwen2.5-coder

Step 3: VS Code Integration

The integration relies on the Chat: Manage Language Models command. This step registers the local endpoint with the Copilot extension.

1. Open the Command Palette (Ctrl+Shift+P or Cmd+Shift+P).
2. Execute Chat: Manage Language Models.
3. Select Add Models and choose Ollama.
4. VS Code auto-detects http://localhost:11434. If using a remote endpoint, input the secure URL (e.g., https://ollama.internal.domain).

Critical Configuration: Ensure the model entry indicates Tools support. Without tool capability, Copilot cannot execute agent actions or interact with the workspace effectively.

Step 4: Session Targeting

Once configured, you must explicitly direct Copilot to use the local model.

1. Open Copilot Chat.
2. Set the session target to Local.
3. Use the model picker to select your Ollama model (e.g., qwen2.5-coder:14b).

Prompts sent in this session will route to the local API. Verify traffic by monitoring Ollama logs:

journalctl -u ollama -f | grep "POST /v1/chat/completions"

Step 5: Secure Remote Access (Optional)

For homelab or team setups, expose Ollama via a reverse proxy with authentication. Direct exposure of the API is a security risk.

Nginx Configuration with Basic Auth:

upstream ollama_backend {
    server 127.0.0.1:11434;
}

server {
    listen 443 ssl;
    server_name ai-coding.internal;

    ssl_certificate /etc/ssl/certs/ai-coding.crt;
    ssl_certificate_key /etc/ssl/private/ai-coding.key;

    # Enforce authentication
    auth_basic "Restricted AI Access";
    auth_basic_user_file /etc/nginx/.htpasswd;

    location / {
        proxy_pass http://ollama_backend;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Timeout adjustments for long inference
        proxy_read_timeout 300s;
        proxy_send_timeout 300s;
    }
}

Generate the password file:

sudo htpasswd -c /etc/nginx/.htpasswd aiuser

Pitfall Guide

1. Unbound API Exposure

Explanation: Configuring Ollama to bind to 0.0.0.0 without firewall rules or authentication exposes the inference engine to the network. Attackers can exploit this to run unauthorized inference or access internal data. Fix: Always use firewall ACLs, Tailscale/WireGuard tunnels, or reverse proxy authentication. Never expose port 11434 to the public internet.

2. VRAM Miscalculation

Explanation: Attempting to load a 32B model on a GPU with 12GB VRAM results in out-of-memory errors or severe performance degradation due to CPU offloading. Fix: Consult VRAM requirements before pulling models. Use nvidia-smi to monitor memory usage. If VRAM is insufficient, switch to smaller models like phi4-mini or higher quantization levels.

3. Inline Autocomplete Expectation

Explanation: BYOK with local models does not fully support inline autocomplete. Developers may expect ghost text suggestions similar to cloud Copilot. Fix: Manage expectations. Local BYOK is optimized for Chat and Agent mode. Inline autocomplete remains a cloud-centric feature in the current implementation.

4. Session Target Drift

Explanation: VS Code may default to cloud models even after local configuration, causing confusion about where prompts are routed. Fix: Always verify the session target is set to Local in the Copilot Chat interface. Check Ollama logs to confirm requests are arriving locally.

5. Tool Calling Failure

Explanation: The model must support tool calling for agent mode to function. If the model entry in VS Code does not show "Tools", agent capabilities will be disabled. Fix: Ensure the selected model supports tool calling. Verify the model configuration in Chat: Manage Language Models indicates tool support.

6. Quantization Ignorance

Explanation: Different quantization levels (e.g., Q4_K_M vs Q8_0) affect model size and quality. Users may pull a model variant that doesn't match their hardware constraints. Fix: Specify quantization when pulling models if needed. Understand that lower quantization reduces VRAM usage but may impact code generation quality.

7. Nginx Timeout Errors

Explanation: Local inference can be slower than cloud APIs, especially on CPU or lower-end GPUs. Default Nginx timeouts may cut off long-running requests. Fix: Increase proxy_read_timeout and proxy_send_timeout in the Nginx configuration to accommodate inference latency.

Production Bundle

Action Checklist

• Verify VS Code (1.113+), Copilot Extension (0.41.0+), and Ollama (0.18.3+) versions.
• Install Ollama and enable the systemd service.
• Assess hardware VRAM and select an appropriate model variant.
• Pull the model and verify availability via ollama list.
• Configure VS Code via Chat: Manage Language Models to add Ollama.
• Confirm the model entry shows "Tools" support for agent capabilities.
• Set Copilot Chat session target to Local and select the model.
• Test inference with a code explanation prompt and monitor Ollama logs.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Enterprise Compliance	Local BYOK	Zero data egress; full control over inference.	High hardware CapEx.
Solo Developer, Low Spec	Cloud Copilot	No hardware requirements; full feature set.	Monthly subscription.
Hybrid Workflow	BYOK for Chat, Cloud for Autocomplete	Balances privacy for sensitive tasks with productivity features.	Mixed costs.
Team Homelab	Remote Ollama + Nginx Proxy	Centralized inference; shared GPU resources.	Hardware + Network setup.

Configuration Template

Ollama Systemd Override for Remote Binding:

# /etc/systemd/system/ollama.service.d/override.conf
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_KEEP_ALIVE=24h"

Secure Nginx Reverse Proxy:

upstream ollama_backend {
    server 127.0.0.1:11434;
}

server {
    listen 443 ssl;
    server_name ai-coding.internal;

    ssl_certificate /etc/ssl/certs/ai-coding.crt;
    ssl_certificate_key /etc/ssl/private/ai-coding.key;

    auth_basic "Restricted AI Access";
    auth_basic_user_file /etc/nginx/.htpasswd;

    location / {
        proxy_pass http://ollama_backend;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        proxy_read_timeout 300s;
        proxy_send_timeout 300s;
    }
}

Quick Start Guide

1. Install Ollama: Run curl -fsSL https://ollama.com/install.sh | sh and start the service.
2. Pull Model: Execute ollama pull qwen2.5-coder:14b to load the coding model.
3. Configure VS Code: Use Chat: Manage Language Models to add Ollama and verify tool support.
4. Activate Local Session: Set Copilot Chat target to Local and select the model.
5. Verify: Send a test prompt and confirm responses originate from the local model via Ollama logs.