ously.
3. Aggregation Worker: Processes events, applies business logic, and updates counters.
4. Billing Integration: Syncs aggregated usage with the payment provider.
Architecture Decisions
- Async-First Metering: Never block the request path for billing. Use a fire-and-forget pattern with reliable delivery guarantees.
- Idempotency: Every metered event must have a unique identifier to prevent double-counting during retries.
- Decimal Precision: Use fixed-point arithmetic or decimal libraries for all monetary and high-precision metric calculations. Floating-point math is unacceptable in billing.
- Metric Definitions as Code: Define billable metrics in a schema-driven configuration to allow dynamic updates without deployment.
Step-by-Step Implementation
1. Define Billable Metrics Schema
Create a TypeScript interface for metric definitions. This allows the system to handle multiple metric types (count, bytes, duration).
export type MetricType = 'COUNT' | 'BYTES' | 'DURATION_MS' | 'COMPUTE_UNIT';
export interface MetricDefinition {
id: string;
name: string;
type: MetricType;
aggregation: 'SUM' | 'MAX' | 'AVG';
unit: string;
precision: number; // Decimal places
}
export const METRICS: MetricDefinition[] = [
{ id: 'api_call', name: 'API Request', type: 'COUNT', aggregation: 'SUM', unit: 'calls', precision: 0 },
{ id: 'data_volume', name: 'Data Transfer', type: 'BYTES', aggregation: 'SUM', unit: 'bytes', precision: 0 },
{ id: 'gpu_seconds', name: 'GPU Compute', type: 'DURATION_MS', aggregation: 'SUM', unit: 'ms', precision: 3 },
];
2. Instrumentation Middleware
Implement middleware that extracts context, validates the tenant, and publishes events to the bus.
import { Request, Response, NextFunction } from 'express';
import { EventPublisher } from './event-publisher';
import { v4 as uuidv4 } from 'uuid';
export interface MonetizationContext {
tenantId: string;
apiKey: string;
planId: string;
}
export const monetizeMiddleware = (
metrics: MetricDefinition[],
publisher: EventPublisher
) => {
return async (req: Request, res: Response, next: NextFunction) => {
const startTime = process.hrtime.bigint();
// Capture response size
const originalSend = res.send;
res.send = function (body: any) {
const responseSize = Buffer.byteLength(JSON.stringify(body));
res.send = originalSend;
res.send(body);
// Publish metering event asynchronously
const event = {
eventId: uuidv4(),
timestamp: Date.now(),
tenantId: (req as any).tenantId, // Set by auth middleware
metric: 'api_call',
value: 1,
metadata: {
endpoint: req.path,
method: req.method,
status: res.statusCode,
responseBytes: responseSize,
},
};
// Fire and forget with retry logic in publisher
publisher.publish('usage.events', event).catch(err => {
console.error('Metering publish failed:', err);
// Alerting mechanism here
});
if (metrics.find(m => m.id === 'data_volume')) {
publisher.publish('usage.events', {
...event,
eventId: uuidv4(),
metric: 'data_volume',
value: responseSize,
}).catch(console.error);
}
};
next();
};
};
3. Aggregation Worker
The worker consumes events, deduplicates using the event bus's delivery guarantees or internal state, and updates a time-series database or Redis counters.
import { Consumer } from 'kafkajs';
import Decimal from 'decimal.js';
export class AggregationWorker {
private seenEvents = new Set<string>(); // In prod, use Redis with TTL
async consume(consumer: Consumer) {
await consumer.subscribe({ topic: 'usage.events' });
await consumer.run({
eachMessage: async ({ message }) => {
const event = JSON.parse(message.value!.toString());
// Idempotency check
if (this.seenEvents.has(event.eventId)) return;
this.seenEvents.add(event.eventId);
// Aggregation logic
const key = `usage:${event.tenantId}:${event.metric}:${this.getBillingCycle()}`;
// Atomic increment in Redis or DB
await this.store.increment(key, new Decimal(event.value));
// Trigger quota check if threshold approached
await this.checkQuota(event.tenantId, event.metric);
},
});
}
private getBillingCycle(): string {
const now = new Date();
return `${now.getFullYear()}-${now.getMonth() + 1}`;
}
}
4. Billing Integration
Sync aggregated usage to the billing provider (e.g., Stripe, Chargebee) via webhooks or API calls at the end of the billing cycle or in real-time for pre-paid models.
export class BillingSync {
async syncUsage(tenantId: string, stripeCustomerId: string) {
const usage = await this.fetchAggregatedUsage(tenantId);
for (const metric of usage) {
await stripe.billing.meterEvents.create({
event_name: metric.metricName,
payload: {
stripe_customer_id: stripeCustomerId,
value: metric.totalValue.toString(),
},
});
}
}
}
Rationale
- Kafka/Event Bus: Decouples the ingestion rate from processing speed. Handles backpressure during traffic spikes.
- Decimal.js: Ensures financial accuracy.
0.1 + 0.2 !== 0.3 in IEEE 754; Decimal avoids this.
- Idempotency: Critical for distributed systems. If the worker crashes after processing but before acknowledging, the message is redelivered. The
seenEvents check prevents double billing.
- Metadata Enrichment: Storing endpoint and status allows for advanced analytics and dispute resolution.
Pitfall Guide
1. Synchronous Billing Calls
Mistake: Calling the billing API inside the request handler.
Impact: Increases p99 latency by 100ms+. If the billing API is slow or down, your product becomes unavailable.
Best Practice: Always use async publishing. The request must succeed or fail based on business logic, not billing availability.
2. Race Conditions in Counters
Mistake: Using non-atomic operations to increment counters (e.g., read-modify-write).
Impact: Lost counts under concurrent load. Revenue leakage scales with traffic.
Best Practice: Use atomic increments in Redis (INCRBY) or database transactions.
3. Floating-Point Arithmetic
Mistake: Using standard numbers for currency or precise metrics.
Impact: Rounding errors accumulate. A customer might be billed $0.0000001 less per request, leading to disputes or compliance issues.
Best Practice: Use Decimal.js or store values as integers (e.g., cents or milliwatts) and format only at presentation.
4. Webhook Reliability
Mistake: Assuming webhooks from billing providers arrive exactly once and in order.
Impact: Duplicate charges or missing updates.
Best Practice: Implement idempotency keys on all incoming webhooks. Verify signatures. Handle out-of-order events by comparing timestamps.
5. Timezone and Billing Cycle Edge Cases
Mistake: Hardcoding UTC or ignoring DST changes for monthly cycles.
Impact: Customers billed for partial months incorrectly. Support tickets spike at month-end.
Best Practice: Store all timestamps in UTC. Use a library like date-fns or luxon for cycle calculations. Define billing cycles explicitly in configuration.
6. Metric Drift
Mistake: Changing the definition of a metric (e.g., counting requests vs. counting bytes) without migration.
Impact: Historical data becomes incomparable. Pricing models break.
Best Practice: Version metric definitions. When changing logic, create a new metric ID and migrate pricing, keeping the old metric for historical reporting.
Mistake: Logging sensitive data in metering events or exposing usage details in public dashboards.
Impact: GDPR/CCPA violations. Competitors can infer traffic patterns.
Best Practice: Sanitize metadata. Hash PII. Ensure usage data is accessible only to the tenant and authorized admins.
Production Bundle
Action Checklist
Decision Matrix
| Scenario | Recommended Approach | Why | Cost Impact |
|---|
| Early-Stage API | Flat Subscription + Stripe Billing | Fastest time-to-market. Low engineering overhead. | Low |
| High-Volume Data API | Granular Usage-Based + Async Aggregation | Aligns cost with value. Handles variable load efficiently. | Medium |
| Enterprise SaaS | Hybrid Model + Contract Management | Supports negotiated terms, committed spend, and overage. | High |
| Marketplace Asset | Revenue Share + Marketplace Payouts | Leverages marketplace trust and handles multi-party payments. | Medium |
| Real-Time Compute | Pre-paid Credits + Burst Quotas | Prevents runaway costs for users. Ensures cash flow. | Medium |
Configuration Template
Use this JSON schema to define your monetization configuration. This can be loaded dynamically by your application.
{
"version": "1.0",
"currency": "USD",
"billingCycle": "MONTHLY",
"metrics": [
{
"id": "api_requests",
"displayName": "API Requests",
"type": "COUNT",
"precision": 0,
"aggregation": "SUM",
"pricing": {
"model": "TIERED",
"tiers": [
{ "upTo": 10000, "pricePerUnit": 0.00 },
{ "upTo": 100000, "pricePerUnit": 0.001 },
{ "upTo": null, "pricePerUnit": 0.0005 }
]
}
},
{
"id": "storage_gb",
"displayName": "Storage (GB)",
"type": "BYTES",
"precision": 3,
"aggregation": "MAX",
"pricing": {
"model": "FLAT",
"pricePerUnit": 0.10
}
}
],
"quotas": {
"rateLimit": {
"requestsPerSecond": 100,
"burstMultiplier": 1.5
}
}
}
Quick Start Guide
-
Initialize Project:
mkdir monetization-engine && cd monetization-engine
npm init -y
npm install express kafkajs decimal.js uuid stripe
npm install -D typescript @types/node @types/express
tsc --init
-
Create Config:
Save the Configuration Template as config/monetization.json.
-
Deploy Infrastructure:
Run a local Kafka broker and Redis instance using Docker Compose:
version: '3'
services:
kafka:
image: confluentinc/cp-kafka:latest
ports: ["9092:9092"]
environment:
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
redis:
image: redis:alpine
ports: ["6379:6379"]
docker-compose up -d
-
Run Aggregation Worker:
Start the worker process to consume events and update Redis counters.
ts-node src/worker.ts
-
Instrument API:
Add the monetizeMiddleware to your Express routes. Test by sending requests and verifying events appear in Kafka and counters increment in Redis.
This architecture provides a scalable, accurate, and flexible foundation for monetizing digital assets, enabling data-driven pricing strategies while maintaining system reliability.