prioritization-config.yaml

ing the prioritization with identical inputs must yield identical outputs to ensure auditability.

Step-by-Step Implementation

1. Define Input Schema: Establish a strict schema for feature metadata. Required fields include business value, user impact, engineering cost, technical risk, and strategic alignment.
2. Develop Scoring Algorithm: Implement a weighted scoring function. The score should penalize cost and risk while rewarding value and alignment.
3. Integrate with Issue Tracker: Use webhooks or scheduled jobs to fetch issue data, run the scorer, and update labels/rankings in the tracker.
4. Calibration Loop: Quarterly, analyze the delta between predicted and actual metrics to adjust weights.

Code Example: TypeScript Prioritization Engine

The following TypeScript implementation provides a type-safe, extensible prioritization engine. It supports dynamic weight configuration and calculates a normalized score.

// src/prioritization/types.ts

export interface FeatureMetrics {
  id: string;
  title: string;
  
  // Value Dimensions (1-10 scale)
  businessValue: number;
  userImpact: number;
  strategicAlignment: number;
  
  // Cost & Risk Dimensions
  engineeringCost: number; // Estimated hours
  technicalRisk: number;   // 1-10 scale
  dependencyCount: number; // Number of cross-team dependencies
  
  // Metadata
  ageInDays: number;
  tags: string[];
}

export interface PrioritizationConfig {
  weights: {
    businessValue: number;
    userImpact: number;
    strategicAlignment: number;
    engineeringCost: number;
    technicalRisk: number;
    dependencyCount: number;
    ageDecay: number; // Penalty for aging items
  };
  thresholds: {
    maxCostForQuickWin: number;
    highRiskThreshold: number;
  };
}

export interface ScoreResult {
  featureId: string;
  score: number;
  classification: 'quick-win' | 'major-project' | 'money-pit' | 'fill-in';
  breakdown: Record<string, number>;
}

// src/prioritization/scorer.ts

export class FeaturePrioritizer {
  constructor(private config: PrioritizationConfig) {
    this.validateConfig();
  }

  private validateConfig(): void {
    const totalWeight = Object.values(this.config.weights).reduce((a, b) => a + b, 0);
    if (Math.abs(totalWeight - 1.0) > 0.01) {
      throw new Error('Configuration weights must sum to 1.0');
    }
  }

  public calculateScore(feature: FeatureMetrics): ScoreResult {
    const breakdown: Record<string, number> = {};
    
    // Normalize inputs to 0-1 range where necessary
    const normalizedCost = this.normalizeCost(feature.engineeringCost);
    const normalizedRisk = feature.technicalRisk / 10;
    const normalizedDeps = Math.min(feature.dependencyCount / 5, 1);
    const agePenalty = Math.min(feature.ageInDays / 90, 1);

    // Calculate weighted components
    breakdown.businessValue = feature.businessValue * this.config.weights.businessValue;
    breakdown.userImpact = feature.userImpact * this.config.weights.userImpact;
    breakdown.strategicAlignment = feature.strategicAlignment * this.config.weights.strategicAlignment;
    breakdown.cost = -normalizedCost * this.config.weights.engineeringCost;
    breakdown.risk = -normalizedRisk * this.config.weights.technicalRisk;
    breakdown.dependencies = -normalizedDeps * this.config.weights.dependencyCount;
    breakdown.age = -agePenalty * this.config.weights.ageDecay;

    const totalScore = Object.values(breakdown).reduce((a, b) => a + b, 0);
    const classification = this.classifyFeature(feature, totalScore);

    return {
      featureId: feature.id,
      score: Math.max(0, totalScore), // Floor at 0
      classification,
      breakdown
    };
  }

  private normalizeCost(cost: number): number {
    // Logarithmic normalization to handle exponential cost variance
    return Math.log10(cost + 1) / 4; // Assumes max cost ~10,000 hours
  }

  private classifyFeature(feature: FeatureMetrics, score: number): ScoreResult['classification'] {
    const isLowCost = feature.engineeringCost <= this.config.thresholds.maxCostForQuickWin;
    const isHighValue = feature.businessValue >= 7;
    const isHighRisk = feature.technicalRisk >= this.config.thresholds.highRiskThreshold;

    if (isLowCost && isHighValue) return 'quick-win';
    if (!isLowCost && isHighValue && !isHighRisk) return 'major-project';
    if (isLowCost && !isHighValue && !isHighRisk) return 'fill-in';
    return 'money-pit'; // High cost, low value, or high risk
  }

  public rankFeatures(features: FeatureMetrics[]): ScoreResult[] {
    return features
      .map(f => this.calculateScore(f))
      .sort((a, b) => b.score - a.score);
  }
}

Usage Example

import { FeaturePrioritizer, PrioritizationConfig, FeatureMetrics } from './prioritization';

const config: PrioritizationConfig = {
  weights: {
    businessValue: 0.3,
    userImpact: 0.2,
    strategicAlignment: 0.15,
    engineeringCost: 0.15,
    technicalRisk: 0.1,
    dependencyCount: 0.05,
    ageDecay: 0.05
  },
  thresholds: {
    maxCostForQuickWin: 40,
    highRiskThreshold: 7
  }
};

const prioritizer = new FeaturePrioritizer(config);

const backlog: FeatureMetrics[] = [
  {
    id: 'FEAT-101',
    title: 'Implement SSO for Enterprise',
    businessValue: 9,
    userImpact: 5,
    strategicAlignment: 10,
    engineeringCost: 120,
    technicalRisk: 6,
    dependencyCount: 2,
    ageInDays: 45,
    tags: ['security', 'enterprise']
  },
  {
    id: 'FEAT-102',
    title: 'Update Button Colors',
    businessValue: 2,
    userImpact: 3,
    strategicAlignment: 1,
    engineeringCost: 4,
    technicalRisk: 1,
    dependencyCount: 0,
    ageInDays: 120,
    tags: ['ui']
  }
];

const ranked = prioritizer.rankFeatures(backlog);
console.log(ranked);
// Output: FEAT-101 ranked higher due to strategic alignment and value, 
// despite higher cost. FEAT-102 classified as 'fill-in' but low score.

Pitfall Guide

1. Ignoring Technical Debt in Cost Estimates:

   • Mistake: Engineering estimates often reflect "happy path" implementation time, ignoring refactoring, testing infrastructure, and documentation required for the feature.
   • Mitigation: Apply a "Complexity Multiplier" based on the module's existing technical debt score. If a feature touches a high-debt module, increase the cost estimate by 1.5x to 2x.

2. Static Weight Drift:

   • Mistake: Setting weights once and never revisiting them. Business priorities shift; a model optimized for growth in Q1 may be disastrous for retention in Q4.
   • Mitigation: Schedule quarterly weight reviews. Use the calibration loop to analyze which features delivered value and adjust weights to favor similar inputs.

3. The "HiPPO" Override:

   • Mistake: Allowing the Highest Paid Person's Opinion to override the prioritization score without a documented exception process.
   • Mitigation: Implement an "Exception Protocol." Overrides require a signed risk acknowledgment and a post-mortem commitment. The system should flag overridden items for audit.

4. False Precision in Scoring:

   • Mistake: Treating a score of 7.85 as significantly better than 7.82. This leads to analysis paralysis and false confidence.
   • Mitigation: Group scores into tiers (e.g., P0, P1, P2) rather than ranking every single item linearly. Focus on relative ordering within tiers, not absolute decimal differences.

5. Neglecting Dependency Costs:

   • Mistake: Scoring features in isolation without accounting for cross-team dependencies. A feature with low internal cost but high external dependency cost can block multiple teams.
   • Mitigation: Include a dependencyCount or dependencyRisk metric in the schema. Features requiring coordination with >2 other teams should carry a penalty or require a "Dependency Resolution" spike before scoring.

6. Optimizing for Short-Term Value Only:

   • Mistake: The scoring model consistently deprioritizes foundational work, platform improvements, or experimental features because their immediate ROI is hard to quantify.
   • Mitigation: Allocate a fixed budget percentage (e.g., 20%) for "Strategic Enablers" that bypass the standard scoring model. Or, introduce a "Platform Value" weight that rewards infrastructure improvements.

7. Lack of Post-Release Validation:

   • Mistake: Scoring features based on predicted value but never measuring actual value. The model never learns.
   • Mitigation: Integrate analytics tools. 30 days post-release, automatically fetch usage metrics and update the feature's "Actual Value" field. Use this data to recalibrate future predictions.

Production Bundle

Action Checklist

• Audit Backlog Metadata: Ensure all active features have populated fields for value, cost, risk, and alignment. Flag items with missing data for grooming.
• Define Weight Configuration: Collaborate with product and engineering leads to define initial weights based on current strategic goals. Document the rationale.
• Deploy Scoring Engine: Implement the prioritization service or CLI tool. Integrate with your issue tracker via API/webhooks.
• Establish Calibration Cadence: Schedule a quarterly review to analyze prediction accuracy and adjust weights. Assign an owner for this process.
• Configure Exception Protocol: Define the process for overriding scores. Create a tracking label for "Score Overridden" items.
• Set Up Post-Release Analytics: Configure automated collection of usage metrics for shipped features to feed the feedback loop.
• Train Stakeholders: Conduct a workshop to explain the scoring model, inputs, and outputs. Ensure alignment on the "why" behind the rankings.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Early-Stage Startup	RICE + Manual Grooming	Speed and flexibility outweigh precision. Over-engineering the prioritization system consumes scarce resources.	Low
Scale-Up / Growth Phase	Calibrated Vector Engine	Volume of features increases; manual grooming becomes a bottleneck. Data-driven decisions reduce waste and align teams.	Medium
Enterprise / Regulated	Weighted Scoring + Compliance Gates	Risk management and auditability are paramount. Scoring must integrate with compliance checks and security reviews.	High
Maintenance Mode	Cost-First Prioritization	Focus shifts to efficiency and stability. Prioritize items that reduce maintenance cost or technical debt.	Low
Crisis / Incident Response	Dynamic Triage Override	Standard prioritization is suspended. Immediate severity and impact drive decisions. System must support rapid re-ranking.	Variable

Configuration Template

Copy this YAML configuration to initialize your prioritization engine. Adjust weights and thresholds based on your organization's context.

# prioritization-config.yaml
version: "1.0"
metadata:
  last_updated: "2024-05-20"
  owner: "engineering-leads"
  strategy: "Q2 Growth & Retention"

weights:
  business_value: 0.30
  user_impact: 0.20
  strategic_alignment: 0.15
  engineering_cost: 0.15
  technical_risk: 0.10
  dependency_count: 0.05
  age_decay: 0.05

thresholds:
  quick_win_max_cost_hours: 40
  high_risk_score: 7
  aging_penalty_start_days: 60
  aging_penalty_max_days: 180

calibration:
  enabled: true
  review_frequency: "quarterly"
  feedback_sources:
    - "analytics_platform"
    - "sprint_retrospectives"
    - "post_release_metrics"

overrides:
  allow_hippo_override: true
  require_risk_acknowledgment: true
  audit_tag: "score-override"

Quick Start Guide

1. Initialize Configuration: Create prioritization-config.yaml using the template above. Set weights that reflect your current quarter's goals.
2. Export Backlog: Run a query in your issue tracker to export active features to a JSON file. Ensure the export includes all fields defined in the FeatureMetrics interface.
3. Run Scorer: Execute the prioritization CLI:

   npx @codcompass/prioritizer run \
     --config ./prioritization-config.yaml \
     --input ./backlog-export.json \
     --output ./ranked-backlog.json
   

4. Review Output: Inspect ranked-backlog.json. Verify the top-ranked items align with strategic expectations. Check classifications (quick-win, major-project, etc.).
5. Apply Labels: Use the generated scores to update labels in your issue tracker. For example, apply priority/P0 to items in the top 10% of the score distribution.

This approach transforms feature prioritization from a subjective debate into a repeatable, data-driven engineering practice. By implementing a calibrated scoring system, organizations can maximize engineering ROI, reduce technical debt accumulation, and ensure that development efforts directly contribute to measurable business outcomes.