What Happens in 2 Milliseconds: Anatomy of a Single HTTP Request Through a Production WAF

c" "time" )

// RiskScore represents the accumulated threat level of a request. type RiskScore int

const ( BlockThreshold RiskScore = 100 SoftBlockThreshold RiskScore = 75 )

// SecurityGateway manages the inspection pipeline. type SecurityGateway struct { ipEngine *IPIntelligence rateLimiter *SlidingRateLimiter headerCheck *HeaderAnalyzer payloadIns *PayloadInspector }

// RequestContext holds state for a single request traversal. type RequestContext struct { IP string Score RiskScore IsBlocked bool BlockReason string }

// Intercept processes the request through the pipeline. func (gw *SecurityGateway) Intercept(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { ctx := &RequestContext{ IP: extractClientIP(r), }

	// Stage 1: IP Intelligence (O(1) lookups)
	gw.ipEngine.Evaluate(ctx)
	if ctx.IsBlocked {
		gw.reject(w, r, ctx)
		return
	}

	// Stage 2: Behavioral Heuristics
	gw.rateLimiter.Check(ctx)
	gw.headerCheck.Evaluate(r, ctx)
	
	// Early exit if score is already critical
	if ctx.Score >= BlockThreshold {
		gw.reject(w, r, ctx)
		return
	}

	// Stage 3: Payload Inspection (Expensive, deferred)
	gw.payloadIns.Evaluate(r, ctx)

	// Stage 4: Decision
	if ctx.Score >= BlockThreshold {
		gw.reject(w, r, ctx)
		return
	}

	next.ServeHTTP(w, r)
})

}

func (gw *SecurityGateway) reject(w http.ResponseWriter, r *http.Request, ctx *RequestContext) { http.Error(w, "Forbidden", http.StatusForbidden) // Emit metrics here }

func extractClientIP(r *http.Request) string { // Production: Check X-Forwarded-For, X-Real-IP, or RemoteAddr return r.RemoteAddr }

#### Stage 1: IP Intelligence with Temporal Decay

IP reputation must account for the transient nature of threat actors. Scores should decay over time to prevent stale data from causing false positives. The engine uses hash maps for O(1) access and maintains a background eviction routine.

```go
type IPIntelligence struct {
	mu         sync.RWMutex
	blocklist  map[string]struct{}
	torExits   map[string]struct{}
	cidrs      []*net.IPNet
	historical map[string]*ipSignal
}

type ipSignal struct {
	score    RiskScore
	lastSeen time.Time
}

func (e *IPIntelligence) Evaluate(ctx *RequestContext) {
	e.mu.RLock()
	defer e.mu.RUnlock()

	if _, ok := e.blocklist[ctx.IP]; ok {
		ctx.IsBlocked = true
		ctx.BlockReason = "IP blocklist"
		return
	}

	if _, ok := e.torExits[ctx.IP]; ok {
		ctx.Score += 70
	}

	// CIDR check for datacenter ranges
	parsedIP := net.ParseIP(ctx.IP)
	for _, cidr := range e.cidrs {
		if cidr.Contains(parsedIP) {
			ctx.Score += 40
			break
		}
	}

	// Apply decay to historical signals
	if sig, exists := e.historical[ctx.IP]; exists {
		hoursSince := time.Since(sig.lastSeen).Hours()
		// Half-life decay: score halves every 24 hours
		decayFactor := math.Pow(0.5, hoursSince/24.0)
		adjustedScore := RiskScore(float64(sig.score) * decayFactor)
		ctx.Score += adjustedScore
	}
}

Rationale: Tor exit nodes receive a high base score (70) due to the high correlation with malicious activity, but they do not trigger an immediate hard block unless combined with other signals. This allows legitimate privacy-conscious users to pass if no other risk factors are present. CIDR checks add moderate risk for datacenter origins, which are common sources of automated attacks.

Stage 2: Behavioral Heuristics

Real browsers exhibit consistent header patterns. Automation libraries often omit standard headers or use identifiable User-Agent strings. Rate limiting uses a sliding window to prevent boundary exploits.

type SlidingRateLimiter struct {
	mu      sync.Mutex
	windows map[string]*requestWindow
	limit   int
	window  time.Duration
}

type requestWindow struct {
	timestamps []int64 // Nanosecond precision for compact storage
}

func (rl *SlidingRateLimiter) Check(ctx *RequestContext) {
	rl.mu.Lock()
	defer rl.mu.Unlock()

	now := time.Now().UnixNano()
	cutoff := now - rl.window.Nanoseconds()

	win, exists := rl.windows[ctx.IP]
	if !exists {
		win = &requestWindow{}
		rl.windows[ctx.IP] = win
	}

	// Prune expired entries in-place
	valid := 0
	for _, ts := range win.timestamps {
		if ts > cutoff {
			win.timestamps[valid] = ts
			valid++
		}
	}
	win.timestamps = win.timestamps[:valid]

	if len(win.timestamps) >= rl.limit {
		ctx.Score += 25
	} else {
		win.timestamps = append(win.timestamps, now)
	}
}

type HeaderAnalyzer struct {
	automationSignatures []string
}

func (h *HeaderAnalyzer) Evaluate(r *http.Request, ctx *RequestContext) {
	ua := strings.ToLower(r.UserAgent())
	for _, sig := range h.automationSignatures {
		if strings.Contains(ua, sig) {
			ctx.Score += 30
			break
		}
	}

	if r.Header.Get("Accept") == "" {
		ctx.Score += 15
	}

	if r.Method == http.MethodPost && r.Header.Get("Referer") == "" {
		ctx.Score += 10
	}

	// Hard block for header injection attempts
	for _, values := range r.Header {
		for _, v := range values {
			if strings.ContainsAny(v, "\r\n") {
				ctx.IsBlocked = true
				ctx.BlockReason = "Header injection detected"
				return
			}
		}
	}
}

Rationale: The sliding window prevents the "boundary burst" exploit where attackers send requests just before and after a fixed window resets. Header analysis adds risk for missing browser artifacts. Header injection (\r\n) is treated as a hard block because it indicates an attempt to poison downstream caches or split HTTP responses, which is never legitimate.

Stage 3: Payload Inspection

Payload inspection is the most expensive stage. It must normalize inputs to defeat encoding evasion and use pre-compiled regex patterns to minimize CPU overhead.

type PayloadInspector struct {
	rules []*InspectionRule
}

type InspectionRule struct {
	ID       string
	Pattern  *regexp.Regexp
	Severity int // 1-4; 4 is critical
	Target   TargetType
}

type TargetType int

const (
	TargetBody TargetType = 1 << iota
	TargetURL
)

func (p *PayloadInspector) Evaluate(r *http.Request, ctx *RequestContext) {
	body, _ := io.ReadAll(r.Body)
	r.Body = io.NopCloser(bytes.NewReader(body))

	normBody := normalizePayload(body)
	normURL := normalizePayload([]byte(r.URL.RawQuery + r.URL.Path))

	for _, rule := range p.rules {
		var target []byte
		if rule.Target&TargetBody != 0 {
			target = normBody
		}
		if rule.Target&TargetURL != 0 {
			target = append(target, normURL...)
		}

		if rule.Pattern.Match(target) {
			ctx.Score += RiskScore(rule.Severity * 15)
			if rule.Severity == 4 {
				ctx.IsBlocked = true
				ctx.BlockReason = rule.ID
				return
			}
		}
	}
}

func normalizePayload(input []byte) []byte {
	// Double URL decode to handle evasion
	s, _ := url.QueryUnescape(string(input))
	s, _ = url.QueryUnescape(s)
	return []byte(strings.ToLower(s))
}

Rationale: Normalization performs double URL decoding to catch attacks that use nested encoding to bypass simple pattern matching. Regex patterns are pre-compiled and stored in the InspectionRule struct; compiling regex per request would cause severe CPU degradation. Critical severity rules (Severity 4) trigger immediate hard blocks regardless of the accumulated score, as they indicate high-confidence attacks like SQL injection or command execution.

Pitfall Guide

1. Fixed-Window Rate Limiting

   • Explanation: Fixed windows reset at absolute time boundaries, allowing attackers to burst requests at the end of one window and the start of the next, effectively doubling the allowed rate.
   • Fix: Implement a sliding window algorithm that tracks timestamps within a rolling duration, as shown in the SlidingRateLimiter.

2. Per-Request Regex Compilation

   • Explanation: Calling regexp.Compile or regexp.MustCompile inside the request handler causes unnecessary CPU allocation and garbage collection pressure.
   • Fix: Compile all patterns during initialization and reuse the *regexp.Regexp instances across requests.

3. Ignoring Encoding Layers

   • Explanation: Attackers frequently use URL encoding, double encoding, or Unicode normalization to obfuscate payloads. Inspecting raw bytes misses these variants.
   • Fix: Implement recursive normalization that decodes inputs multiple times and converts to a canonical case before matching.

4. Unbounded Memory Growth

   • Explanation: Storing historical IP scores or rate limit windows without eviction leads to memory leaks, especially under high traffic with diverse source IPs.
   • Fix: Implement background goroutines to evict entries with decayed scores below a threshold or expired timestamps. Use TTL-based maps where appropriate.

5. Hard Blocking on Single Signals

   • Explanation: Blocking requests based on a single heuristic (e.g., missing User-Agent) increases false positives, as legitimate clients may have non-standard configurations.
   • Fix: Use a scoring model that requires multiple corroborating signals before blocking. Reserve hard blocks for critical severity rules or explicit blocklists.

6. Blocking Legitimate Automation

   • Explanation: API clients and internal services often use automation libraries that trigger heuristic rules, causing service disruption.
   • Fix: Implement allowlists for known API paths or internal service IPs. Lower thresholds for endpoints that are expected to receive programmatic traffic.

7. Header Injection Blind Spots

   • Explanation: Failing to inspect header values for control characters allows HTTP response splitting and cache poisoning attacks.
   • Fix: Scan all header values for \r\n sequences and treat them as hard blocks, regardless of the risk score.

Production Bundle

Action Checklist

• Pre-compile all regex patterns during application startup to avoid runtime compilation overhead.
• Implement temporal decay for IP reputation scores to prevent stale data from causing false positives.
• Use sliding window rate limiting to prevent boundary burst exploits.
• Normalize payloads with double URL decoding and case conversion before inspection.
• Configure dynamic thresholds that adjust based on traffic patterns and time of day.
• Set up background eviction for historical IP data to manage memory usage.
• Enable shadow mode to test new rules without blocking traffic during deployment.
• Monitor false positive rates and adjust scoring weights based on operational feedback.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High-traffic API endpoint	Allowlist known clients; lower heuristic thresholds	Prevents disruption to legitimate automation	Low latency, minimal CPU
Public-facing web app	Full pipeline with strict thresholds	Maximizes protection against diverse threats	Moderate latency, higher CPU for inspection
Internal service mesh	IP-based allowlist only	Reduces overhead for trusted traffic	Near-zero latency
New rule deployment	Shadow mode with logging	Validates rule accuracy without blocking	No latency impact, storage cost for logs

Configuration Template

gateway:
  thresholds:
    block: 100
    soft_block: 75
    shadow: 50
  
  ip_intelligence:
    tor_exit_score: 70
    datacenter_score: 40
    decay_half_life_hours: 24
    eviction_threshold: 5
  
  rate_limiting:
    window_seconds: 10
    max_requests: 60
  
  payload_inspection:
    normalization:
      url_decode_passes: 2
      lowercase: true
    rules:
      - id: "SQLI_CRITICAL"
        pattern: "\\bor\\b\\s+['\"]?\\w+['\"]?\\s*=\\s*['\"]?\\w+['\"]?"
        severity: 4
        target: body
      - id: "XSS_REFLECTED"
        pattern: "<script[\\s/>]|javascript\\s*:"
        severity: 4
        target: body|url

Quick Start Guide

1. Define Rules: Create a configuration file with regex patterns, severity levels, and scoring thresholds tailored to your application's risk profile.
2. Initialize Gateway: Instantiate the SecurityGateway with pre-compiled rules and configure IP intelligence sources (blocklists, Tor exit nodes).
3. Wrap Handlers: Apply the Intercept middleware to your HTTP handlers or router to route traffic through the pipeline.
4. Deploy in Shadow Mode: Start with shadow mode enabled to log decisions without blocking. Monitor logs for false positives and adjust scores.
5. Enable Enforcement: Once validation is complete, switch to enforcement mode. Continuously monitor latency metrics and false positive rates.