MVP definition and validation

// validation-engine.ts
export interface Hypothesis {
  id: string;
  description: string;
  exposureEvent: string;
  actionEvent: string;
  outcomeEvent: string;
  successThreshold: number; // decimal (0.30 = 30%)
  evaluationWindowMs: number;
}

export interface ValidationEvent {
  hypothesisId: string;
  userId: string;
  timestamp: number;
  type: 'exposure' | 'action' | 'outcome';
  metadata?: Record<string, unknown>;
}

export class ValidationEngine {
  private hypotheses: Map<string, Hypothesis> = new Map();
  private eventBuffer: ValidationEvent[] = [];

  register(hypothesis: Hypothesis): void {
    this.hypotheses.set(hypothesis.id, hypothesis);
  }

  track(event: ValidationEvent): void {
    this.eventBuffer.push({ ...event, timestamp: Date.now() });
  }

  evaluate(): Record<string, { status: 'pass' | 'fail' | 'pending'; conversion: number; sampleSize: number }> {
    const results: Record<string, { status: 'pass' | 'fail' | 'pending'; conversion: number; sampleSize: number }> = {};

    for (const [id, hyp] of this.hypotheses) {
      const exposure = this.eventBuffer.filter(e => e.hypothesisId === id && e.type === 'exposure');
      const action = this.eventBuffer.filter(e => e.hypothesisId === id && e.type === 'action');
      const outcome = this.eventBuffer.filter(e => e.hypothesisId === id && e.type === 'outcome');

      const now = Date.now();
      const win

dowExposure = exposure.filter(e => now - e.timestamp <= hyp.evaluationWindowMs); const windowOutcome = outcome.filter(e => now - e.timestamp <= hyp.evaluationWindowMs);

  const conversion = windowExposure.length > 0 ? windowOutcome.length / windowExposure.length : 0;
  const status = conversion >= hyp.successThreshold ? 'pass' : conversion < hyp.successThreshold * 0.5 ? 'fail' : 'pending';

  results[id] = { status, conversion, sampleSize: windowExposure.length };
}

return results;

}

flush(): void { this.eventBuffer = []; } }

### Step 3: Gate Delivery with Feature Flags
Validation requires controlled exposure. The MVP should not be rolled out to 100% of traffic. Use a feature flag system to route a validation cohort, collect signals, and evaluate before scaling.

```typescript
// flag-router.ts
import { ValidationEngine } from './validation-engine';

export class ValidationRouter {
  constructor(private engine: ValidationEngine, private flagService: any) {}

  async shouldExpose(userId: string, hypothesisId: string): Promise<boolean> {
    const enabled = await this.flagService.isEnabled(`mvp-${hypothesisId}`, userId);
    if (enabled) {
      this.engine.track({ hypothesisId, userId, type: 'exposure' });
    }
    return enabled;
  }
}

Step 4: Run Structured Validation Cycles

Validation is not a one-time check. It follows a fixed cadence:

1. Deploy hypothesis-gated feature to 5–10% cohort
2. Run for 3–5 business days
3. Evaluate against thresholds
4. Pass → scale to 50% → instrument secondary metrics
5. Fail → kill feature → document learnings
6. Pending → extend window or refine action constraint

Architecture decisions supporting this flow:

• Decoupled telemetry: Validation events are emitted independently of business logic to prevent coupling validation metrics to UI state.
• Immutable hypothesis definitions: Hypotheses are registered at startup and cannot be mutated at runtime, ensuring evaluation consistency.
• Time-bounded evaluation: Windows prevent stale data from skewing conversion rates.
• Threshold-based routing: Success/failure gates control flag rollout, not manual approval.

Pitfall Guide

1. Treating MVP as a Beta Release

Betas ship incomplete features to gather usability feedback. MVPs ship complete hypotheses to gather market signal. Mixing the two produces noisy data: users report UI friction instead of value validation. Best practice: Isolate validation cohorts. Beta feedback routes to UX tickets; MVP validation routes to hypothesis evaluation.

2. Optimizing for Code Coverage Instead of Signal Coverage

Teams measure test coverage, branch coverage, and deployment frequency. These metrics confirm code quality, not market viability. Best practice: Track hypothesis exposure rate, action completion rate, and outcome conversion rate. Treat these as first-class engineering metrics.

3. Ignoring Cohort Retention in Favor of Vanity Metrics

Daily active users and session length inflate during early rollout due to novelty effects. Retention curves reveal whether the MVP solves a recurring job. Best practice: Measure D1, D3, D7 retention for the validation cohort. If D7 < 20%, the hypothesis is failing regardless of initial spike.

4. Skipping Explicit Success/Failure Thresholds

Vague goals like “see if users like it” cannot be evaluated. Without decimal thresholds and sample size requirements, teams rationalize failure as “early days.” Best practice: Define minimum viable conversion, minimum sample size (e.g., 500 exposures), and evaluation window before deployment.

5. Over-Engineering the Validation Infrastructure

Building custom analytics pipelines, real-time dashboards, and ML-driven prediction models before validating the core hypothesis wastes capacity. Best practice: Use event buffering, simple threshold evaluation, and flag-based gating. Scale telemetry only after the first hypothesis passes.

6. Assuming Validation is a Phase, Not a Continuous Loop

Treating MVP validation as a pre-launch gate ignores that market conditions, user behavior, and competitive landscapes shift. Best practice: Run validation cycles quarterly for core features. Register new hypotheses when metrics drift >15% from baseline.

7. Misaligning Technical Architecture with Validation Goals

Monolithic deployments, coupled state management, and synchronous API chains make it impossible to isolate hypothesis impact. Best practice: Use modular feature boundaries, event-driven communication, and read-model separation. Validation requires the ability to expose, measure, and roll back independently.

Production Bundle

Action Checklist

• Define single riskiest hypothesis with explicit success threshold and evaluation window
• Instrument exposure, action, and outcome events using a decoupled validation engine
• Register hypothesis in configuration before deployment; never mutate at runtime
• Gate rollout to 5–10% cohort using feature flags; isolate validation traffic
• Run validation for 3–5 business days; collect D1/D3/D7 retention alongside conversion
• Evaluate against thresholds; pass → scale, fail → kill, pending → refine constraint
• Document learnings in hypothesis registry; update architecture based on signal strength

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Pre-seed startup with <100 users	Validation-First MVP with mock backend	Speed of learning outweighs infrastructure cost; mock APIs reduce engineering hours by 60%	Low initial cost, high learning velocity
Enterprise internal tool	Hypothesis-gated rollout with strict cohort isolation	Compliance and change management require controlled exposure; validation prevents costly rework	Medium cost, risk reduction justifies investment
B2C SaaS with existing traffic	Feature-flagged MVP with A/B exposure and retention tracking	Existing user base provides rapid signal; retention metrics prevent false positives from novelty	Low incremental cost, high scaling confidence

Configuration Template

{
  "validation": {
    "hypotheses": [
      {
        "id": "hyp-csv-export-v1",
        "description": "Users who export CSV within 3 clicks will upgrade at 30% rate within 7 days",
        "exposureEvent": "mvp.csv.export.shown",
        "actionEvent": "mvp.csv.export.completed",
        "outcomeEvent": "billing.upgrade.completed",
        "successThreshold": 0.30,
        "evaluationWindowMs": 604800000,
        "minSampleSize": 500,
        "flagKey": "mvp-hyp-csv-export-v1",
        "initialCohortPercent": 8
      }
    ],
    "evaluation": {
      "cadenceHours": 24,
      "retentionMetrics": ["D1", "D3", "D7"],
      "autoScaleThreshold": 0.35,
      "autoKillThreshold": 0.12,
      "pendingExtensionDays": 2
    },
    "telemetry": {
      "bufferSize": 1000,
      "flushIntervalMs": 30000,
      "deduplication": true,
      "schemaVersion": "1.0.0"
    }
  }
}

Quick Start Guide

1. Install dependencies: npm install @codcompass/validation-engine flag-sdk (or use the provided TypeScript module directly)
2. Register hypothesis: Load the configuration template and call engine.register(hypothesis) at application startup
3. Instrument events: Emit exposure, action, and outcome events using engine.track() at the corresponding user interactions
4. Run evaluation: Schedule engine.evaluate() every 24 hours; route results to your CI/CD pipeline or dashboard
5. Gate rollout: Connect evaluation status to your feature flag provider; scale or kill based on pass/fail/pending states

Validation is not a product management exercise. It is an engineering discipline that converts uncertainty into measurable signal. Define the riskiest assumption, instrument the loop, enforce thresholds, and let data dictate scale. Ship less code, learn faster, and scale only what the market validates.