Difficulty

Intermediate

Read Time

9 min

Deploying ClearML as an AWS SageMaker Alternative on Ubuntu

By Codcompass Team·2026-07-04·9 min read

Self-Hosted MLOps Orchestration: Building a Unified ML Lifecycle Stack on Ubuntu

Current Situation Analysis

Modern machine learning teams face a persistent fragmentation problem. Experiment tracking lives in one tool, pipeline orchestration in another, model serving in a third, and hyperparameter optimization often requires custom scripting or expensive cloud add-ons. When teams attempt to consolidate these workflows, they typically gravitate toward managed cloud platforms. While convenient, these services introduce steep operational costs, data egress fees, and vendor lock-in that become unsustainable as experiment volume scales.

The core misunderstanding lies in treating MLOps as a collection of isolated utilities rather than a continuous data flow. Teams frequently stitch together open-source components only to spend more time maintaining integrations than training models. Conversely, self-hosting a unified platform is often dismissed as operationally heavy, despite modern container orchestration reducing the overhead to a single compose file.

Data from infrastructure cost benchmarks shows that mid-sized ML teams (5-15 practitioners) running 50+ concurrent experiments monthly typically spend $800-$2,500 on managed cloud ML services. A self-hosted equivalent running on equivalent bare-metal or VM infrastructure averages $150-$400 in fixed compute costs, while retaining full control over artifact storage, network topology, and credential management. The trade-off is no longer cost versus convenience; it's architectural control versus operational debt.

WOW Moment: Key Findings

Consolidating the ML lifecycle into a single self-hosted stack fundamentally changes how teams interact with model artifacts and compute resources. The following comparison highlights the operational shift:

Dimension	Managed Cloud ML Platform	Self-Hosted Unified Stack
Monthly Baseline Cost	$800–$2,500+ (compute + storage + API)	$150–$400 (fixed VM/bare-metal)
Data Residency	Cloud region-dependent, egress fees apply	Fully on-prem/air-gapped, zero egress
Feature Coverage	Fragmented across separate services	Single API surface for tracking, pipelines, HPO, serving
Setup Complexity	Low initial, high long-term integration debt	Moderate initial, near-zero integration overhead
GPU Utilization	Pay-per-second, often underutilized	Persistent queue routing, higher saturation

This finding matters because it decouples ML scalability from cloud billing cycles. Teams can run continuous hyperparameter sweeps, version pipelines, and deploy serving endpoints without triggering unexpected invoice spikes. The unified architecture also enables deterministic reproducibility: every experiment, dataset snapshot, and model artifact lives in the same storage backend, eliminating cross-service synchronization failures.

Core Solution

Deploying a self-hosted MLOps stack requires three coordinated layers: persistent storage services, a reverse proxy for secure external access, and compute agents that execute workloads. The following implementation uses Docker Compose for orchestration, Traefik for TLS termination and routing, and Python-based agents for workload execution.

1. Host Preparation & Storage Initialization

Elasticsearch, which powers the metadata backend, requires an elevated virtual memory map limit. Without this, the container will fail to allocate memory during index creation.

sudo tee /etc/sysctl.d/99-ml-ops.conf << EOF
vm.max_map_count=524288
EOF
sudo sysctl --system
sudo systemctl restart docker

Next, provision persistent volumes with the correct UID/GID. The official stack expects user 1000:1000 to avoid permission denials on mounted paths.

sudo mkdir -p /opt/ml-platform/{elastic,mongo-db,mongo-cfg,redis,filestore,logs,config}
sudo chown -R 1000:1000 /opt/ml-platform

2. Core Service Deployment

Pull the official orchestration manifest and adjust network boundaries. Isolating frontend and backend traffic prevents accidental exposure of internal services.

mkdir -p ~/ml-platform && cd ~/ml-platform
curl -fsSL https://raw.githubusercontent.com/clearml/clearml-server/master/docker/docker-compose.yml -o docker-compose.yml

Edit docker-compose.yml to remove direct port mappings and define explicit

bridge networks:

networks:
  ml_backend:
    name: ml_backend_net
    driver: bridge
  ml_frontend:
    name: ml_frontend_net
    driver: bridge

Define public endpoints in .env:

ML_WEB_HOST=https://ui.ml-platform.example.com
ML_API_HOST=https://api.ml-platform.example.com
ML_FILES_HOST=https://artifacts.ml-platform.example.com

Launch the stack:

docker compose up -d
docker compose logs --tail 30 -f

3. Secure Reverse Proxy Configuration

Traefik handles TLS termination and routes subdomain traffic to the appropriate internal service. Create a dedicated project directory and initialize the certificate store:

mkdir -p ~/ml-platform/proxy/letsencrypt
touch ~/ml-platform/proxy/letsencrypt/acme.json
chmod 600 ~/ml-platform/proxy/letsencrypt/acme.json

Proxy compose manifest:

services:
  proxy:
    image: traefik:v3.6
    container_name: ml_proxy
    command:
      - "--log.level=WARN"
      - "--providers.file.filename=/etc/traefik/routes.yml"
      - "--entryPoints.http.address=:80"
      - "--entryPoints.https.address=:443"
      - "--entryPoints.http.http.redirections.entrypoint.to=https"
      - "--certificatesResolvers.letsencrypt.acme.httpChallenge.entryPoint=http"
      - "--certificatesResolvers.letsencrypt.acme.email=ops@example.com"
      - "--certificatesResolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "./letsencrypt:/letsencrypt"
      - "./routes.yml:/etc/traefik/routes.yml:ro"
    networks:
      - ml_frontend_net
    restart: unless-stopped

networks:
  ml_frontend_net:
    name: ml_frontend_net
    external: true

Routing rules (routes.yml):

http:
  routers:
    ui-router:
      rule: "Host(`ui.ml-platform.example.com`)"
      entryPoints: [https]
      service: ui-svc
      tls: {certResolver: letsencrypt}
    api-router:
      rule: "Host(`api.ml-platform.example.com`)"
      entryPoints: [https]
      service: api-svc
      tls: {certResolver: letsencrypt}
    artifacts-router:
      rule: "Host(`artifacts.ml-platform.example.com`)"
      entryPoints: [https]
      service: artifacts-svc
      tls: {certResolver: letsencrypt}
  services:
    ui-svc:
      loadBalancer:
        servers: [{url: "http://ml-webserver:80"}]
    api-svc:
      loadBalancer:
        servers: [{url: "http://ml-apiserver:8008"}]
    artifacts-svc:
      loadBalancer:
        servers: [{url: "http://ml-fileserver:8081"}]

Start the proxy and verify certificate issuance:

docker compose -f ~/ml-platform/proxy/docker-compose.yml up -d
docker logs ml_proxy 2>&1 | grep -i "certificate obtained"

4. Agent Registration & Workload Execution

Compute agents poll queues and execute tasks in isolated environments. Create a dedicated virtual environment to prevent dependency conflicts:

mkdir -p ~/ml-worker && cd ~/ml-worker
python3 -m venv worker_env
source worker_env/bin/activate
pip install clearml-agent
clearml-agent init

Paste the generated credentials block when prompted. The agent will store them in ~/clearml.conf. Launch the daemon:

clearml-agent daemon --queue production-queue --detached

For GPU workloads, specify device indices and ensure the NVIDIA Container Toolkit is installed:

clearml-agent daemon --gpus 0,1 --queue gpu-queue --detached

Verify registration in the web interface under Workers & Queues.

5. Experiment Tracking & Artifact Management

Initialize a tracking task, log hyperparameters, and persist model artifacts. This example uses California housing data to demonstrate scalar logging and artifact versioning.

from clearml import Task
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.metrics import mean_squared_error
import joblib

experiment = Task.init(
    project_name="Production ML",
    task_name="Housing Price Baseline",
    tags=["regression", "baseline"]
)

params = {"n_estimators": 150, "learning_rate": 0.05, "max_depth": 4}
experiment.connect(params)

data = fetch_california_housing()
X_train, X_test, y_train, y_test = train_test_split(
    data.data, data.target, test_size=0.2, random_state=2024
)

model = GradientBoostingRegressor(**params).fit(X_train, y_train)
rmse = mean_squared_error(y_test, model.predict(X_test), squared=False)
experiment.get_logger().report_scalar("Validation", "RMSE", value=rmse, iteration=1)

joblib.dump(model, "housing_gbr_model.pkl")
experiment.upload_artifact("final_model", "housing_gbr_model.pkl")
experiment.close()

6. Pipeline Orchestration

Define reproducible workflows using function-based steps. Parameters are passed through the pipeline controller, ensuring deterministic execution.

from clearml import PipelineController

def ingest_raw_data(remote_url: str):
    import pandas as pd
    from clearml import StorageManager
    local_path = StorageManager.get_local_copy(remote_url=remote_url)
    return pd.read_csv(local_path)

def split_dataset(df: pd.DataFrame, ratio: float = 0.2):
    from sklearn.model_selection import train_test_split
    target = df["target"]
    features = df.drop(columns=["target"])
    return train_test_split(features, target, test_size=ratio, random_state=42)

def train_regressor(data_tuple):
    from sklearn.linear_model import Ridge
    X_train, X_test, y_train, y_test = data_tuple
    return Ridge(alpha=1.0).fit(X_train, y_train)

if __name__ == "__main__":
    pipeline = PipelineController(
        project="Production ML",
        name="Data-to-Model Workflow",
        version="2.1",
        add_pipeline_tags=True
    )
    pipeline.add_parameter("source_url", "https://example.com/datasets/housing.csv")
    pipeline.add_function_step(
        "load_data", ingest_raw_data,
        function_kwargs={"remote_url": "${pipeline.source_url}"},
        function_return=["dataset"]
    )
    pipeline.add_function_step(
        "partition", split_dataset,
        function_kwargs={"df": "${load_data.dataset}"},
        function_return=["split_data"]
    )
    pipeline.add_function_step(
        "fit_model", train_regressor,
        function_kwargs={"data_tuple": "${partition.split_data}"},
        function_return=["trained_estimator"]
    )
    pipeline.start_locally(run_pipeline_steps_locally=True)

7. Hyperparameter Optimization Sweep

Automate search spaces using the optimizer controller. Define parameter ranges, objective metrics, and concurrency limits.

from clearml import Task
from clearml.automation import (
    HyperParameterOptimizer,
    UniformIntegerParameterRange,
    DiscreteParameterRange,
    RandomSearch,
)

base_task = Task.get_tasks(project_name="Production ML", task_name="Housing Price Baseline")[-1]
optimizer_task = Task.init(
    project_name="Production ML",
    task_name="HPO Sweep",
    task_type=Task.TaskTypes.optimizer
)

optimizer = HyperParameterOptimizer(
    base_task_id=base_task.id,
    hyper_parameters=[
        UniformIntegerParameterRange("General/n_estimators", min_value=50, max_value=300, step_size=25),
        DiscreteParameterRange("General/max_depth", values=[3, 5, 7]),
    ],
    objective_metric_title="Validation",
    objective_metric_series="RMSE",
    objective_metric_sign="min",
    optimizer_class=RandomSearch,
    max_number_of_concurrent_tasks=3,
    total_max_jobs=9,
)
optimizer.start()
optimizer.wait()
best_params = optimizer.get_top_experiments(1)[0].get_parameters_as_dict()
print("Optimal configuration:", best_params)

8. Model Serving Deployment

ClearML Serving wraps models with a Triton-compatible backend. Clone the serving repository, configure credentials, and initialize the endpoint.

cd ~/ml-platform
git clone https://github.com/clearml/clearml-serving.git
pip install clearml-serving
clearml-serving create --name "production-inference"

Configure the serving environment:

ML_WEB_HOST="https://ui.ml-platform.example.com"
ML_API_HOST="https://api.ml-platform.example.com"
ML_FILES_HOST="https://artifacts.ml-platform.example.com"
ML_API_ACCESS_KEY="YOUR_ACCESS_KEY"
ML_API_SECRET_KEY="YOUR_SECRET_KEY"
ML_SERVING_TASK_ID="GENERATED_TASK_ID"

Launch the inference stack:

cd ~/ml-platform/clearml-serving/docker
docker compose --env-file .env -f docker-compose-triton.yml up -d

Register the trained model artifact, assign it to the endpoint, and validate latency through the provided REST interface.

Pitfall Guide

Pitfall	Explanation	Fix
Elasticsearch memory limit not persisted	`vm.max_map_count` resets after reboot, causing index creation failures	Add the `sysctl` directive to `/etc/sysctl.d/` and verify with `sysctl vm.max_map_count`
Traefik network isolation mismatch	Proxy and core services reside on different Docker networks, resulting in 502 Bad Gateway	Explicitly attach both compose projects to the same external bridge network
Agent daemon blocking CI/CD pipelines	Running `clearml-agent daemon` in foreground halts automation scripts	Always use `--detached` flag and monitor via `clearml-agent status`
HPO queue starvation	Concurrent tasks exceed queue capacity, causing optimizer timeouts	Set `max_number_of_concurrent_tasks` ≤ available agent slots, or scale agent replicas
Pipeline step serialization errors	Functions reference uninstalled packages or relative imports not available in agent environment	Use `clearml-agent` with explicit `--requirements` flag or bake dependencies into a custom Docker image
Serving model version conflicts	Updating an endpoint without versioning causes inference failures for active clients	Enable model versioning in the serving UI and route traffic using weighted deployments
Credential exposure in version control	`clearml.conf` or `.env` files committed to Git leak access keys	Add credential files to `.gitignore`, use environment injection, or leverage secret management tools

Production Bundle

Action Checklist

DNS propagation: Verify A records for UI, API, and artifact subdomains resolve correctly
Memory tuning: Confirm vm.max_map_count=524288 persists across reboots
Network isolation: Validate frontend/backend Docker networks are correctly attached to all services
TLS validation: Check certificate expiration dates and enforce HTTPS-only routing
Queue routing: Assign agents to named queues matching workload types (CPU/GPU/Inference)
Artifact retention: Configure fileserver cleanup policies to prevent unbounded storage growth
Monitoring: Deploy Prometheus/Grafana or use built-in health endpoints for service uptime tracking

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Small team (<5 ML engineers)	Single-node Docker Compose	Simplifies maintenance, reduces orchestration overhead	Low ($150-$300/mo VM)
GPU-heavy HPO workloads	Dedicated GPU agents + queue routing	Maximizes hardware utilization, prevents CPU/GPU contention	Moderate (GPU instance pricing)
Strict data compliance	Air-gapped deployment + local fileserver	Eliminates cloud egress, satisfies regulatory requirements	Fixed (on-prem hardware)
High-availability serving	Multi-replica Triton containers + load balancer	Ensures zero-downtime deployments and horizontal scaling	Higher (redundant compute)

Configuration Template

# docker-compose.yml (core services excerpt)
services:
  apiserver:
    image: allegroai/clearml-apiserver:latest
    volumes:
      - /opt/ml-platform/config:/opt/clearml/config:ro
      - /opt/ml-platform/data:/opt/clearml/data
    networks:
      - ml_backend_net
      - ml_frontend_net
    environment:
      - CLEARML_API_HOST=${ML_API_HOST}
      - CLEARML_FILES_HOST=${ML_FILES_HOST}
      - CLEARML_WEB_HOST=${ML_WEB_HOST}
    restart: unless-stopped

networks:
  ml_backend_net:
    external: true
  ml_frontend_net:
    external: true

# .env
ML_WEB_HOST=https://ui.ml-platform.example.com
ML_API_HOST=https://api.ml-platform.example.com
ML_FILES_HOST=https://artifacts.ml-platform.example.com
ML_API_ACCESS_KEY=YOUR_ACCESS_KEY
ML_API_SECRET_KEY=YOUR_SECRET_KEY

Quick Start Guide

Provision host: Spin up an Ubuntu 22.04+ instance with Docker, Compose, and DNS A records pointing to the host IP.
Initialize storage: Run the sysctl command, create /opt/ml-platform directories, and set ownership to 1000:1000.
Launch stack: Pull the official compose manifest, configure .env with your subdomains, and execute docker compose up -d.
Attach proxy: Deploy Traefik with the provided routing rules, verify TLS certificate issuance, and confirm UI accessibility.
Register worker: Create a Python venv, install clearml-agent, initialize credentials, and start the daemon on a named queue.

The stack is now ready to ingest experiments, orchestrate pipelines, execute hyperparameter sweeps, and serve models without cloud vendor dependencies.

🎉 Mid-Year Sale — Unlock Full Article

Base plan from just $4.99/mo or $49/yr

7-day free trial · Cancel anytime · 30-day money-back