experiment assignment, and monetary context when applicable.
interface GrowthEvent {
eventId: string;
userId: string | null;
sessionId: string;
timestamp: number;
eventType: 'page_view' | 'click' | 'signup' | 'purchase' | 'refund';
channel: string;
campaign?: string;
experiment?: {
id: string;
variant: string;
timestamp: number;
};
monetary?: {
amount: number;
currency: string;
status: 'completed' | 'pending' | 'refunded';
};
consent: {
analytics: boolean;
marketing: boolean;
};
}
Step 2: Real-Time Attribution Router
Replace static last-touch models with a decay-weighted multi-touch router. This assigns credit across touchpoints using a configurable decay function, preventing early-funnel channels from monopolizing revenue attribution.
type Touchpoint = { channel: string; timestamp: number };
function calculateAttribution(
touchpoints: Touchpoint[],
conversionTimestamp: number,
decayFactor: number = 0.85
): Record<string, number> {
const sorted = touchpoints.sort((a, b) => b.timestamp - a.timestamp);
const credits: Record<string, number> = {};
sorted.forEach((tp, index) => {
const daysToConversion = (conversionTimestamp - tp.timestamp) / (1000 * 60 * 60 * 24);
const weight = Math.pow(decayFactor, daysToConversion);
credits[tp.channel] = (credits[tp.channel] || 0) + weight;
});
const total = Object.values(credits).reduce((sum, w) => sum + w, 0);
// Normalize to percentages
return Object.fromEntries(
Object.entries(credits).map(([ch, w]) => [ch, w / total])
);
}
Step 3: Experiment Orchestrator with Bandit Routing
Replace static A/B splits with a Thompson Sampling multi-armed bandit. This dynamically shifts traffic toward higher-converting variants while maintaining exploration.
interface VariantStats {
conversions: number;
impressions: number;
}
class BanditExperiment {
private variants: Map<string, VariantStats>;
private alpha: number = 1; // Prior
constructor(variantIds: string[]) {
this.variants = new Map(
variantIds.map(id => [id, { conversions: 0, impressions: 0 }])
);
}
assignVariant(): string {
const samples = new Map<string, number>();
for (const [variant, stats] of this.variants) {
// Beta distribution sampling approximation
const alpha = stats.conversions + this.alpha;
const beta = stats.impressions - stats.conversions + this.alpha;
samples.set(variant, this.betaSample(alpha, beta));
}
return Array.from(samples.entries()).sort((a, b) => b[1] - a[1])[0][0];
}
recordOutcome(variant: string, converted: boolean): void {
const stats = this.variants.get(variant)!;
stats.impressions++;
if (converted) stats.conversions++;
}
private betaSample(alpha: number, beta: number): number {
// Production: replace with a proper beta sampling library
// This is a simplified approximation for demonstration
const x = Math.random();
return x; // Placeholder; use `beta-distribution` npm package in production
}
}
Step 4: Revenue Reconciliation Layer
Growth metrics must reconcile with actual financial data. Implement a sink that matches experiment assignments and attribution credits to completed transactions, filtering out refunds, chargebacks, and trial conversions that never monetize.
interface RevenueRecord {
transactionId: string;
userId: string;
amount: number;
currency: string;
status: 'completed' | 'refunded' | 'failed';
attribution: Record<string, number>;
experimentVariant: string;
}
function reconcileGrowthRevenue(records: RevenueRecord[]): Record<string, number> {
return records
.filter(r => r.status === 'completed')
.reduce((acc, r) => {
const channelShare = Object.entries(r.attribution).reduce((sum, [ch, pct]) => {
return sum + r.amount * pct;
}, 0);
acc[r.experimentVariant] = (acc[r.experimentVariant] || 0) + channelShare;
return acc;
}, {} as Record<string, number>);
}
Architecture Decisions & Rationale
- Decoupled Ingestion: Events flow through a message queue (e.g., Kafka, SQS) to isolate high-throughput ingestion from downstream processing. This prevents traffic spikes from blocking experiment routing.
- Idempotent Processing: Every event carries a deterministic
eventId. Deduplication is enforced at the sink layer to prevent double-counting in attribution and revenue calculations.
- Consent-Aware Routing: The
consent field gates telemetry routing. Marketing/analytics events are dropped or anonymized based on regional compliance requirements, preventing legal exposure.
- Stateless Experiment Service: The bandit router maintains state in a fast key-value store (Redis/DynamoDB) with periodic checkpointing. This enables horizontal scaling and zero-downtime deployments.
- Financial Sink Separation: Revenue reconciliation runs on a separate pipeline with strict idempotency and audit logging. Growth metrics never override accounting data; they annotate it.
Pitfall Guide
-
Misconfigured Attribution Windows: Using a fixed 30-day window across all channels ignores channel latency. Paid search converts in hours; organic content converts in weeks. Apply channel-specific decay parameters or dynamic windowing based on historical conversion velocity.
-
Ignoring Cohort Decay & LTV Normalization: Optimizing for first-purchase conversion inflates short-term metrics while masking churn. Always normalize experiment results by cohort retention curves and projected LTV. A variant that converts 15% more but churns 40% faster is a net loss.
-
Hardcoding Experiment Thresholds: Setting static lift thresholds (e.g., "roll out at +5% conversion") ignores statistical power and variance. Implement sequential testing with power analysis. Stop experiments only when the credible interval excludes the null hypothesis at your chosen confidence level.
-
Over-Instrumentation & Payload Bloat: Tracking every micro-interaction increases latency, storage costs, and noise. Define a minimum viable telemetry schema. Route high-frequency events (scrolls, hovers) to sampled sinks or batch processors. Keep revenue-critical events synchronous and deduplicated.
-
Failing to Reconcile Refunds & Chargebacks: Growth dashboards that count gross revenue without subtracting returns create false positives. Implement a negative-event pipeline that offsets experiment credits and channel attribution when refunds occur. Revenue must be net, not gross.
-
Privacy Compliance Gaps: Routing tracking pixels or setting third-party cookies without explicit consent violates GDPR/CCPA and triggers browser-level blocking. Gate all non-essential telemetry behind a consent manager. Use first-party routing and server-side enrichment to maintain signal without violating policy.
-
Scaling Experiment State Synchronously: Storing variant assignments in application memory or relational databases creates locking bottlenecks during traffic spikes. Externalize state to a distributed cache with TTL-based expiration and background persistence. Never block request threads on experiment lookups.
Production Best Practices:
- Implement circuit breakers on attribution and experiment services to fall back to deterministic routing during outages.
- Use feature flags to toggle telemetry verbosity without redeploying.
- Maintain an experiment registry with immutable variant definitions, start/end dates, and primary/secondary metrics.
- Run parallel shadow experiments to validate attribution models against historical revenue before full rollout.
Production Bundle
Action Checklist
Decision Matrix
| Scenario | Recommended Approach | Why | Cost Impact |
|---|
| Early-stage SaaS (<10k MAU) | Last-touch + static A/B testing | Low complexity, fast validation, minimal infrastructure overhead | Low initial cost; scales poorly beyond 50k MAU |
| E-commerce with seasonal spikes | Decay-weighted attribution + bandit routing | Handles multi-touch journeys, dynamically shifts traffic during high-conversion windows | Medium infrastructure cost; reduces CAC payback by 25-40% |
| Marketplace with dual-sided traffic | Channel-specific attribution + cohort LTV normalization | Prevents buyer/seller channel cannibalization, aligns growth with network effects | High engineering investment; improves LTV:CAC ratio by 1.8x |
| Enterprise B2B with long sales cycles | Multi-touch Markov attribution + sequential testing | Maps complex decision units, validates lift over extended windows without budget bleed | High compliance/audit cost; reduces sales cycle friction by 30% |
Configuration Template
growth_loop:
attribution:
model: decay_weighted
default_window_days: 30
channel_overrides:
paid_search: { window_days: 7, decay_factor: 0.9 }
organic: { window_days: 90, decay_factor: 0.7 }
email: { window_days: 14, decay_factor: 0.85 }
experimentation:
routing: thompson_sampling
power_threshold: 0.85
min_impressions_per_variant: 1000
fallback: deterministic_round_robin
telemetry:
schema_version: v2.1
deduplication_window_ms: 5000
consent_gating: true
high_frequency_sampling_rate: 0.1
revenue:
reconciliation_mode: net
include_refunds: true
currency_normalization: true
cohort_retention_tracking: true
Quick Start Guide
- Install the core SDK:
npm install @growth/telemetry @growth/experiments @growth/attribution
- Initialize the client with your configuration file:
import { GrowthClient } from '@growth/telemetry'; const client = new GrowthClient({ config: './growth.config.yaml' });
- Instrument your primary conversion event:
client.track('purchase', { userId: 'usr_123', monetary: { amount: 49.99, currency: 'USD', status: 'completed' } });
- Deploy the experiment router alongside your feature flag system:
const variant = experimentRouter.assignVariant('pricing_page_v2'); client.assignExperiment('pricing_page_v2', variant);
- Verify in your dashboard: Confirm event ingestion, attribution credit distribution, and experiment assignment balance within 5 minutes. Enable revenue reconciliation and monitor net lift before full rollout.