CI/CD pipeline best practices

;

interface TestMetric { file: string; durationMs: number; }

export function generateShards( metricsPath: string, targetShards: number ): string[][] { const raw = readFileSync(metricsPath, 'utf-8'); const tests: TestMetric[] = JSON.parse(raw);

// Sort by duration descending to pack largest tests first const sorted = [...tests].sort((a, b) => b.durationMs - a.durationMs); const shards: string[][] = Array.from({ length: targetShards }, () => []); const shardDurations: number[] = Array(targetShards).fill(0);

for (const test of sorted) { // Find shard with lowest total duration const minIndex = shardDurations.indexOf(Math.min(...shardDurations)); shards[minIndex].push(test.file); shardDurations[minIndex] += test.durationMs; }

return shards; }

// Usage in CI script const metrics = join(process.env.GITHUB_WORKSPACE || '.', 'test-durations.json'); const shards = generateShards(metrics, 4);

console.log(JSON.stringify(shards.map((s, i) => ({ shard: i + 1, files: s }))));

This script reads historical test durations, applies a greedy bin-packing algorithm, and outputs balanced shard configurations. Integrating it into the CI runner selection step ensures parallel jobs complete within a narrow time window, eliminating stragglers that inflate pipeline duration.

## Pitfall Guide

1. **Monolithic Job Design**
   Combining lint, test, build, and deploy into a single job creates sequential dependencies and amplifies failure impact. A single flaky test blocks artifact generation. Decompose into independent jobs with explicit dependency graphs using `needs`.

2. **Blind Cache Usage Without Invalidation**
   Caching without content-addressable keys leads to stale dependencies and environment drift. Always hash lockfiles and configuration files. Implement fallback keys that degrade gracefully when primary cache misses occur.

3. **Secrets Sprawl and Hardcoded Tokens**
   Embedding credentials in workflow files or environment variables without rotation creates compliance violations and attack surfaces. Use OIDC, short-lived session tokens, and secret scanning. Never persist tokens between runs.

4. **Skipping Pipeline Observability**
   Running pipelines without telemetry masks performance degradation. Track execution duration, cache hit rates, failure categories, and runner utilization. Integrate with monitoring dashboards to detect regressions before they impact delivery velocity.

5. **Environment Drift Between CI and Staging**
   CI runners often run minimal base images that differ from production or staging. This causes false positives and deployment failures. Use containerized runners with identical base images, or enforce infrastructure-as-code parity across environments.

6. **Unbounded Artifact Retention**
   Storing build outputs without expiration policies inflates storage costs and slows down artifact resolution. Compress outputs, set explicit retention windows (e.g., 7 days for PRs, 30 days for releases), and purge intermediate files automatically.

7. **Over-Engineering with Custom Orchestrators**
   Building custom pipeline controllers instead of leveraging native workflow engines introduces maintenance overhead and security risks. Use platform-native features (GitHub Actions, GitLab CI, Argo Workflows) before introducing external orchestration layers.

## Production Bundle

### Action Checklist
- [ ] Modularize workflows: Split monolithic pipelines into reusable composite actions and parameterized workflows to centralize updates and reduce duplication.
- [ ] Implement content-addressable caching: Hash lockfiles and configuration files to generate deterministic cache keys, reducing redundant dependency resolution.
- [ ] Parallelize independent stages: Run lint, test, security scan, and build steps concurrently using matrix strategies or dynamic sharding.
- [ ] Enforce policy-as-code gates: Integrate dependency scanning, license checks, and configuration validation at the earliest stage to fail fast.
- [ ] Configure artifact lifecycle policies: Compress outputs, set explicit retention windows, and purge intermediate files to control storage costs.
- [ ] Enable pipeline observability: Track execution duration, cache hit rates, failure categories, and runner utilization to detect performance regressions.
- [ ] Rotate secrets via OIDC: Replace long-lived credentials with short-lived federated tokens and enforce automatic rotation on runner initialization.

### Decision Matrix

| Scenario | Recommended Approach | Why | Cost Impact |
|----------|---------------------|-----|-------------|
| Small team (<10 devs), single repo | Native platform workflows with matrix parallelization | Low operational overhead, fast iteration, sufficient for moderate concurrency | Low: Pay-per-minute billing scales with usage |
| Enterprise, multi-repo monorepo | Reusable workflows + self-hosted autoscaling runners + dynamic test sharding | Centralized policy enforcement, predictable performance, handles high concurrency | Medium: Higher runner provisioning, offset by 40–60% compute reduction |
| Compliance-heavy (SOC2, HIPAA) | Policy-as-code gates + signed attestations + isolated runner pools | Enforces audit trails, prevents drift, satisfies regulatory requirements | High: Additional scanning steps and isolated infrastructure, but reduces compliance risk |
| Multi-cloud deployment | Abstracted deployment steps + environment-specific OIDC + artifact promotion | Decouples CI from cloud vendor lock-in, enables consistent promotion across regions | Medium: Slight latency increase from abstraction, offset by reduced vendor dependency |

### Configuration Template

```yaml
name: CI/CD Pipeline
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  NODE_VERSION: '20.x'
  CACHE_KEY: ${{ hashFiles('package-lock.json', 'tsconfig.json') }}

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'
      - run: npm ci
      - run: npm run lint

  test:
    needs: lint
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'
      - run: npm ci
      - run: npm run test:shard -- --shard=${{ matrix.shard }}/4

  security:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'
      - run: npm ci
      - run: npm audit --audit-level=moderate
      - uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          severity: 'CRITICAL,HIGH'

  build:
    needs: [test, security]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'
      - run: npm ci
      - run: npm run build
      - uses: actions/upload-artifact@v4
        with:
          name: build-output
          path: dist/
          retention-days: 7

  deploy:
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: actions/download-artifact@v4
        with:
          name: build-output
          path: dist/
      - run: |
          echo "Deploying to production..."
          # Replace with cloud-specific deployment commands
          # Example: aws s3 sync dist/ s3://my-bucket/ --delete

Quick Start Guide

1. Initialize a repository with a standard package.json, tsconfig.json, and test suite. Ensure npm ci and npm run build succeed locally.
2. Create .github/workflows/ci.yml and paste the Configuration Template. Commit and push to trigger the initial run.
3. Verify parallel execution: Check the Actions tab to confirm lint, test, and security run concurrently. Validate that test shards distribute evenly.
4. Configure caching: Run the pipeline twice. Confirm the second run shows cache hits for npm dependencies and reduced execution time.
5. Enforce protection rules: Navigate to repository settings → Branches → Add rule for main. Require status checks for build and deploy before allowing merges.