Temporal Context for LLM Agents: Capturing User Interactions Beyond Static Snapshots

Current Situation Analysis

Modern AI assistants embedded in enterprise dashboards face a fundamental architectural mismatch: they are designed to reason about static state, but human operators interact with dynamic, sequential workflows. When a developer extracts an accessibility tree or DOM snapshot to feed an LLM, they capture a photograph of the current UI. Users, however, ask questions that require a video. They ask, "What was in that modal I just closed?", "Why did that button fail?", or "Which row did I just edit?"

This gap is routinely overlooked because most teams optimize for static context retrieval. The industry standard has been to serialize the current page structure, strip noise, and inject it into the system prompt. While effective for answering "what is visible now?", it fails completely for retrospective queries. The agent receives a clean DOM tree but zero information about the user's preceding actions or the ephemeral UI components that have already been unmounted.

The cost of this oversight is measurable. In production SaaS environments, 30-40% of user queries to embedded AI assistants reference recent interactions or transient UI states. Feeding a static snapshot to these queries yields hallucinated answers or generic fallback responses. Conversely, dumping full session replay data (like rrweb or LogRocket exports) into a prompt introduces 50-200KB of raw mutation records per minute. At standard LLM pricing, that translates to $0.02-$0.05 per query in token costs, with minimal semantic value for reasoning tasks. The industry needs a middle ground: a lightweight, structured log of user actions that bridges human memory with machine state, without bloating context windows or violating privacy boundaries.

WOW Moment: Key Findings

The breakthrough comes from recognizing that LLMs don't need raw DOM mutations or full visual replays. They need a compressed, semantically rich sequence of user intents. By shifting from structural serialization to interaction tracing, we can reduce context overhead by 85% while increasing query accuracy for retrospective questions.

Approach	Contextual Relevance	Token Overhead	Ephemeral UI Coverage	Implementation Complexity
Static DOM Snapshot	Low (0% for past actions)	~1,200 tokens	None	Low
Full Session Replay (rrweb)	High (visual only)	~15,000+ tokens	Full (but unstructured)	High
Interaction Trace (Proposed)	High (action + row context)	~250-350 tokens	Captured at event time	Medium

This finding matters because it decouples agent context from page structure. The LLM no longer needs to reconstruct user intent from a static tree. Instead, it receives a chronological log of deliberate actions, enriched with the exact data context (e.g., table row identifiers) at the moment of interaction. This enables reliable answers to "what just happened" queries while keeping prompt sizes predictable and cost-efficient.

Core Solution

Building a reliable interaction trace requires three architectural decisions: an extensible event capture strategy, deterministic DOM observation, and token-optimized serialization. The following implementation uses TypeScript and React, but the patterns apply to any framework.

Step 1: Strategy Pattern for Event Sources

Enterprise hosts evolve. Today you capture DOM clicks; tomorrow you might need to ingest host-level navigation events, WebSockets, or custom PubSub signals. Hardcoding a single listener creates technical debt. Instead, implement a strategy interface that standardizes lifecycle management and trace retrieval.

export interface IEventCaptureStrategy {
  initialize(): void;
  teardown(): void;
  retrieveTrace(): InteractionLog;
  clearBuffer(): void;
  onNewEvent?: (entry: InteractionEntry) => void;
}

export interface InteractionEntry {
  timestamp: number;
  actionType: string;
  targetRole: string;
  accessibleName: string;
  dataContext: string;
}

export interface InteractionLog {
  entries: InteractionEntry[];
  windowMs: number;
}

Concrete implementations handle specific domains. A DOM-focused observer, a host-message listener, or a composite strategy that merges multiple sources with priority rules. We use plain classes rather than React hooks for three reasons:

1. Framework Agnosticism: The strategy can run in Web Workers, vanilla JS shells, or non-React micro-frontends without adaptation.
2. Testability: Unit tests can instantiate the class, mock DOM events, and assert on retrieveTrace() without rendering a component tree or managing act() cycles.
3. Composition: A CompositeStrategy can internally manage multiple strategy instances and merge their outputs without wrapper components or context forwarding.

Step 2: Capture-Phase DOM Observation

When an AI assistant runs as a Module Federation remote or micro-frontend, it shares the host's document object. This shared DOM is the foundation for cross-boundary tracking. To capture interactions reliably, attach a listener in the capture phase:

class DomInteractionObserver implements IEventCaptureStrategy {
  private buffer: InteractionEntry[] = [];
  private readonly MAX_BUFFER = 25;
  private observerRef: EventListenerOrEventListenerObject | null = null;

  initialize(): void {
    this.observerRef = this.handleRawEvent.bind(this);
    document.addEventListener('pointerdown', this.observerRef, { capture: true });
  }

  teardown(): void {
    if (this.observerRef) {
      document.removeEventListener('pointerdown', this.observerRef, { capture: true });
    }
  }

  private handleRawEvent(event: PointerEvent): void {
    const target = event.target as HTMLElement;
    const actionable = this.resolveActionableElement(target);
    if (!actionable) return;

    const entry: InteractionEntry = {
      timestamp: Date.now(),
      actionType: 'click',
      targetRole: this.extractRole(actionable),
      accessibleName: this.extractName(actionable),
      dataContext: this.extractRowContext(actionable),
    };

    this.buffer.unshift(entry);
    if (this.buffer.length > this.MAX_BUFFER) {
      this.buffer.pop();
    }

    this.onNewEvent?.(entry);
  }

  // ... implementation details for resolution methods
}

The capture phase ({ capture: true }) is non-negotiable. It fires before the target element's own handlers and before any bubble-phase stopPropagation() calls from the host application. While a host could theoretically blind you with stopImmediatePropagation() at the document level, this is exceptionally rare in production frameworks. Most libraries only interrupt bubbling from within component trees, leaving capture-phase listeners intact.

Step 3: Actionable Element Resolution & Noise Filtering

Raw pointer events fire on every pixel. Logging wrappers, decorative icons, and scroll containers creates signal noise. Resolve the nearest interactive element by walking up the DOM tree and validating against an ARIA role allowlist:

const INTERACTIVE_ROLES = new Set([
  'button', 'link', 'menuitem', 'tab', 'checkbox',
  'radio', 'combobox', 'textbox', 'searchbox', 'switch', 'row',
]);

private resolveActionableElement(start: HTMLElement): HTMLElement | null {
  let current: HTMLElement | null = start;
  while (current) {
    if (current.dataset.assistantPanel === 'true') return null;
    if (current.dataset.noTrack === 'true') return null;

    const role = current.getAttribute('role') || current.tagName.toLowerCase();
    if (INTERACTIVE_ROLES.has(role)) return current;

    current = current.parentElement;
  }
  return null;
}

Apply three additional filters to prevent log pollution:

• Temporal Debounce: Ignore identical targets within a 250ms window to suppress double-clicks and rapid UI toggles.
• Name Validation: Discard elements lacking an aria-label, title, or visible text content. Unlabeled controls provide zero semantic value to the LLM.
• Boundary Guard: Exclude events originating inside the AI panel itself to prevent recursive logging.

Step 4: React State Synchronization

DOM events are synchronous. React state updates are asynchronous. This mismatch creates the classic "one-behind" bug where the UI reads stale trace data. Bridge the gap by triggering a state patch immediately upon event capture:

export function useInteractionTracker(strategy: IEventCaptureStrategy) {
  const [sessionLog, setSessionLog] = useState<InteractionLog>({
    entries: [],
    windowMs: 30000,
  });

  const strategyRef = useRef(strategy);

  useEffect(() => {
    const instance = strategyRef.current;
    instance.onNewEvent = () => {
      setSessionLog({
        entries: instance.retrieveTrace().entries,
        windowMs: instance.retrieveTrace().windowMs,
      });
    };
    instance.initialize();
    return () => instance.teardown();
  }, []);

  return sessionLog;
}

By assigning the callback after instantiation and leveraging a ref, you bypass React's render cycle for the initial capture. The state update fires only when new data arrives, keeping the AI chat component synchronized without unnecessary re-renders.

Step 5: Token-Optimized Serialization

Raw JSON arrays waste tokens on repeated keys, brackets, and structural syntax. Compress the trace into compact natural language before injection. Prioritize row context over landmark context for data-heavy applications. The LLM almost always needs to know which record the user interacted with, not which section of the page.

function serializeTraceForLLM(log: InteractionLog): string {
  const recent = log.entries.slice(0, 15);
  const lines = recent.map((entry) => {
    const time = new Date(entry.timestamp).toLocaleTimeString('en-US', {
      hour12: false,
      hour: '2-digit',
      minute: '2-digit',
      second: '2-digit',
    });
    const context = entry.dataContext
      ? ` (context: ${entry.dataContext})`
      : '';
    return `${time} ${entry.actionType} [${entry.targetRole}] "${entry.accessibleName}"${context}`;
  });

  return `RECENT USER ACTIONS (newest first):\n${lines.join('\n')}`;
}

This format typically consumes 250-350 tokens for a 15-event window. It strips structural noise, preserves chronological order, and embeds the exact data context the LLM needs to reason about the user's workflow.

Pitfall Guide

1. Bubble-Phase Blindness

Explanation: Attaching listeners in the bubble phase ({ capture: false }) means host application handlers can call stopPropagation() before your code executes. The agent misses clicks entirely. Fix: Always use { capture: true }. Verify with a test that dispatches a click on a child element wrapped in a component that calls stopPropagation().

2. The "One-Behind" State Lag

Explanation: React batches state updates. If you only refresh the trace on message send or route change, the AI reads the snapshot from the previous interaction cycle. Fix: Trigger a state patch synchronously via a callback attached to the capture strategy. Ensure the callback updates the context store immediately, not on the next render cycle.

3. Undeclared Interface Callbacks in Compiled TypeScript

Explanation: Assigning strategy.onNewEvent = fn to a class that doesn't explicitly declare onNewEvent as a field can fail silently under certain TS configurations (useDefineForClassFields or Babel class transforms). The method sees undefined. Fix: Always declare optional callback properties explicitly on the class body. onNewEvent?: (entry: InteractionEntry) => void; guarantees deterministic property resolution across all build targets.

4. Landmark-Centric Context Extraction

Explanation: Extracting context from <main>, <aside>, or [role="region"] provides useless spatial data for data tables. The LLM cannot map "in main content" to a specific business record. Fix: Walk up to the nearest <tr>, [role="row"], or grid item. Extract cell text, truncate to 60 characters, and attach it as dataContext. This directly answers "which item" questions.

5. Unbounded Event Buffers

Explanation: Storing every click indefinitely causes memory leaks and token bloat. Long sessions can push trace data past context window limits. Fix: Implement a sliding window (e.g., 25 entries or 30 seconds). Use unshift() for insertion and pop() for eviction. Pass the window size to the serialization function so the LLM knows the temporal scope.

6. Missing Privacy/Compliance Gates

Explanation: Logging every interaction may capture PII, passwords, or sensitive financial data if users type into fields or interact with masked inputs. Fix: Add a data-no-track attribute to sensitive components. Filter out events targeting [type="password"], [inputmode="numeric"], or elements within [data-sensitive="true"]. Audit logs regularly for compliance.

7. Ignoring Framework-Specific Event Delegation

Explanation: Modern frameworks (React, Vue, Angular) often use synthetic event systems or delegate listeners to the root. Direct DOM listeners may fire before framework state updates, capturing stale UI text. Fix: If the host uses synthetic events, consider listening to framework-specific lifecycle hooks or use requestAnimationFrame to defer context extraction until the next paint cycle. Verify that extracted text matches the rendered state.

Production Bundle

Action Checklist

• Define IEventCaptureStrategy interface with initialize, teardown, retrieveTrace, and optional callback
• Implement DomInteractionObserver with capture-phase pointerdown listener
• Build resolveActionableElement walker with ARIA role allowlist and boundary guards
• Add temporal debounce, name validation, and sensitive-data filters
• Create useInteractionTracker hook to bridge sync DOM events to async React state
• Implement serializeTraceForLLM with row-context prioritization and token limits
• Configure sliding window buffer (25 events / 30s) to prevent memory bloat
• Audit host application for stopImmediatePropagation usage and sensitive input fields

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Data-heavy dashboard (tables, grids)	Row-context serialization + capture-phase DOM	LLM needs record-level intent; capture phase bypasses framework delegation	Low (~300 tokens/query)
Form-heavy workflow (multi-step inputs)	Host-level PubSub + field-level masking	DOM clicks lack semantic form state; PubSub provides structured validation events	Medium (~400 tokens/query)
Legacy monolith (no shared DOM)	iFrame postMessage bridge + explicit opt-in	Cannot attach capture listeners; requires explicit host cooperation	High (requires host changes)
Compliance-restricted environment	HostEventSource only + PII scrubbing pipeline	DOM logging violates data residency; host emits sanitized events only	Low (zero DOM overhead)

Configuration Template

// tracker.config.ts
export const TRACKER_CONFIG = {
  buffer: {
    maxEntries: 25,
    windowMs: 30000,
  },
  filters: {
    debounceMs: 250,
    requireAccessibleName: true,
    excludeRoles: ['presentation', 'none', 'img', 'separator'],
    sensitiveSelectors: [
      '[type="password"]',
      '[data-sensitive="true"]',
      '[data-no-track="true"]',
    ],
  },
  serialization: {
    maxEntries: 15,
    contextTruncateChars: 60,
    timeZone: 'UTC',
  },
  strategy: 'dom' as 'dom' | 'host' | 'hybrid',
};

// usage.ts
import { DomInteractionObserver } from './observers/dom';
import { useInteractionTracker } from './hooks/useInteractionTracker';
import { serializeTraceForLLM } from './serializers/llm';
import { TRACKER_CONFIG } from './tracker.config';

const observer = new DomInteractionObserver(TRACKER_CONFIG);
const sessionLog = useInteractionTracker(observer);

// Inject into AI prompt
const promptContext = serializeTraceForLLM(sessionLog);

Quick Start Guide

1. Install & Import: Add the strategy interface and DOM observer to your micro-frontend package. Import useInteractionTracker into your AI chat container.
2. Configure Boundaries: Add data-no-track="true" to password fields, modals containing PII, and the AI panel itself. Verify the observer ignores these targets.
3. Initialize Strategy: Instantiate DomInteractionObserver with your buffer and filter config. Pass it to useInteractionTracker inside your chat component.
4. Serialize & Inject: Call serializeTraceForLLM(sessionLog) before sending messages to your LLM. Append the output to your system prompt under a RECENT USER ACTIONS header.
5. Validate: Open browser DevTools, interact with a data table, and inspect the serialized output. Confirm timestamps are accurate, row context matches the clicked record, and token count stays under 350.