deploy/base/manifest.yaml (Unified deployment descriptor)

eResource(cpu) > this.parseResource(this.MAX_CPU_REQUEST)) { report.errors.push(CPU request ${cpu} exceeds limit ${this.MAX_CPU_REQUEST}); report.valid = false; } if (this.parseResource(memory) > this.parseResource(this.MAX_MEMORY_REQUEST)) { report.errors.push(Memory request ${memory} exceeds limit ${this.MAX_MEMORY_REQUEST}); report.valid = false; } if (this.REQUIRED_HEALTH_CHECK && !container.livenessProbe) { report.warnings.push('Missing liveness probe'); }

return report;

}

validateNomadJob(filePath: string): ValidationReport { const report: ValidationReport = { orchestrator: 'Nomad', valid: true, errors: [], warnings: [] }; const content = fs.readFileSync(filePath, 'utf8'); const job = this.parseNomadHCL(content);

if (!job?.task?.resources?.cpu || !job?.task?.resources?.memory) {
  report.errors.push('Missing resource block');
  report.valid = false;
}

const cpu = job?.task?.resources?.cpu || 0;
const memory = job?.task?.resources?.memory || 0;

if (cpu > 500) report.errors.push(`CPU request ${cpu} exceeds 500 MHz limit`);
if (memory > 512) report.errors.push(`Memory request ${memory} exceeds 512 MB limit`);
if (!job?.task?.config?.check?.http) report.warnings.push('Missing HTTP health check');

return report;

}

private parseResource(value: string): number { const match = value.match(/^(\d+)(m|Mi|Gi)?$/); if (!match) return 0; const num = parseInt(match[1], 10); const unit = match[2]; if (unit === 'm') return num / 1000; if (unit === 'Mi') return num; if (unit === 'Gi') return num * 1024; return num; }

private parseNomadHCL(content: string): any { // Simplified parser for demonstration; production should use hcl2json or nomad parse return { task: { resources: { cpu: 250, memory: 256 }, config: { check: { http: 'http://localhost:8080/health' } } } }; } }

// Usage const validator = new OrchestratorValidator(); const k8sReport = validator.validateKubernetesManifest('./deploy/k8s.yaml'); const nomadReport = validator.validateNomadJob('./deploy/nomad.hcl'); console.log(JSON.stringify([k8sReport, nomadReport], null, 2));

### Architecture Decisions and Rationale

- **OCI Compliance First**: All containers must run on OCI-compliant runtimes. Orchestration should never dictate image format. This preserves portability and prevents vendor lock-in at the build stage.
- **Control Plane Isolation**: Run orchestrator control planes on dedicated nodes or managed services. Co-locating control and data planes causes resource contention during scaling events.
- **GitOps as Single Source of Truth**: Declarative state management eliminates drift. Argo CD, Nomad Sentinel, or native stack commands reconcile desired vs. actual state automatically.
- **Policy Over Configuration**: Use OPA/Gatekeeper, Sentinel, or native admission controllers to enforce resource limits, security contexts, and network policies. Runtime validation catches misconfigurations before deployment.
- **Observability Native to Deployment**: Instrument health checks, metrics endpoints, and distributed tracing at the application layer. Orchestration should consume telemetry, not generate it.

## Pitfall Guide

1. **Treating Kubernetes as Mandatory for Containers**
   Kubernetes solves multi-tenant, complex networking, and advanced scheduling problems. Deploying it for stateless APIs or batch jobs introduces unnecessary control plane overhead. Validate workload topology before defaulting to K8s.

2. **Ignoring CNI/CSI Plugin Overhead**
   Networking and storage plugins consume CPU, memory, and network bandwidth. Calico, Cilium, and Longhorn add latency and complexity. Measure plugin footprint in staging. Prefer host networking or lightweight CNIs for single-tenant deployments.

3. **Confusing Orchestration with CI/CD**
   Orchestration manages runtime state, scaling, and failure recovery. CI/CD handles build, test, and artifact promotion. Mixing responsibilities creates brittle pipelines. Keep deployment manifests declarative and version-controlled.

4. **Underestimating Stateful Workload Complexity**
   Databases and message brokers require consistent scheduling, persistent volumes, and graceful shutdown hooks. Kubernetes StatefulSets and Nomad system jobs handle this differently. Test failover scenarios before production. Avoid stateful workloads on lightweight orchestrators lacking volume orchestration.

5. **Skipping Network Policy Validation**
   Default allow-all networking exposes services to lateral movement. Implement zero-trust microsegmentation early. Test policies with chaos tools like Network Chao or native K8s network policy simulators.

6. **Vendor Lock-in via Managed Add-ons**
   EKS/GKE/AKS managed services often bundle proprietary load balancers, IAM integrations, and monitoring agents. These create migration friction. Abstract cloud-specific components behind interfaces. Use standard Ingress controllers and external DNS.

7. **Delaying Observability Integration**
   Orchestration failures manifest as silent degradation. Deploy metrics collectors, log aggregators, and tracing agents alongside workloads. Configure alerting on scaling latency, pod restart loops, and control plane API latency.

**Best Practices from Production**
- Enforce resource quotas at namespace/job level
- Automate drift detection with GitOps reconciliation
- Use standard health checks (HTTP, gRPC, TCP) instead of custom scripts
- Implement graceful shutdown hooks for stateful services
- Run regular chaos experiments targeting control plane and networking layers
- Document architecture decisions with ADRs tracking orchestrator selection rationale

## Production Bundle

### Action Checklist
- [ ] Profile workload topology: statefulness, scaling pattern, compliance, networking dependencies
- [ ] Benchmark control plane footprint and scaling latency against target orchestrators
- [ ] Implement GitOps pipeline with declarative manifests and environment overrides
- [ ] Enforce resource quotas, network policies, and security contexts via admission controllers
- [ ] Integrate native observability: metrics, logs, traces, and health check endpoints
- [ ] Run chaos experiments targeting control plane, CNI plugins, and storage layers
- [ ] Document architecture decisions with ADRs tracking selection rationale and trade-offs

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| Stateless API cluster with predictable scaling | Nomad | Lower control plane overhead, faster cold starts, simpler networking | 18–25% lower compute cost |
| Multi-tenant SaaS with strict isolation and service mesh | Kubernetes | Native RBAC, network policies, Istio/Linkerd integration, multi-cluster support | 10–15% higher operational cost, justified by compliance |
| Legacy monolith migration with minimal refactoring | Docker Swarm | Minimal configuration, native Docker compatibility, straightforward stack files | Near-zero migration overhead, limited future scaling |
| ML batch jobs and GPU workloads | Nomad or Kubernetes (with operators) | Nomad for pure batch efficiency; K8s for GPU scheduling, dynamic provisioning, and model serving | Nomad: 20% lower idle cost; K8s: higher upfront but better resource packing |

### Configuration Template

```yaml
# deploy/base/manifest.yaml (Unified deployment descriptor)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${APP_NAME}
  labels:
    app: ${APP_NAME}
    environment: ${ENV}
spec:
  replicas: ${REPLICAS}
  selector:
    matchLabels:
      app: ${APP_NAME}
  template:
    metadata:
      labels:
        app: ${APP_NAME}
    spec:
      containers:
        - name: ${APP_NAME}
          image: ${REGISTRY}/${APP_NAME}:${TAG}
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: ${CPU_REQUEST}
              memory: ${MEMORY_REQUEST}
            limits:
              cpu: ${CPU_LIMIT}
              memory: ${MEMORY_LIMIT}
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 5
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 3
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: workload
                    operator: In
                    values: ["${WORKLOAD_TYPE}"]

# deploy/nomad/job.hcl (Nomad equivalent)
job "${APP_NAME}" {
  datacenters = ["${DATACENTER}"]
  type        = "${JOB_TYPE}"

  group "${GROUP_NAME}" {
    count = ${REPLICAS}

    network {
      port "http" {
        static = 8080
      }
    }

    task "app" {
      driver = "docker"

      config {
        image = "${REGISTRY}/${APP_NAME}:${TAG}"
        ports = ["http"]

        check "health" {
          type     = "http"
          path     = "/health"
          interval = "5s"
          timeout  = "2s"
        }
      }

      resources {
        cpu    = ${CPU_REQUEST}
        memory = ${MEMORY_REQUEST}
      }
    }
  }
}

Quick Start Guide

1. Initialize Repository Structure: Create deploy/base/, deploy/k8s/, and deploy/nomad/ directories. Place the unified manifest and orchestrator-specific templates.
2. Install Validation Tooling: Run npm install js-yaml and compile the TypeScript validator. Add it to your CI pipeline as a pre-commit hook.
3. Configure GitOps Reconciliation: Deploy Argo CD for Kubernetes targets or configure Nomad ACL tokens for API-driven deployments. Point the controller to your repository branch.
4. Deploy and Validate: Apply manifests with kubectl apply -f deploy/k8s/ or nomad job run deploy/nomad/job.hcl. Monitor scaling latency and control plane metrics. Validate health checks and network policies before promoting to production.