ry.
- Sweet Spot: The ~100-150ms latency overhead from intent extractors/validators is negligible compared to the LLM generation time, and is offset by reduced token costs and higher first-pass success rates.
Core Solution
The Microsoft Agent Framework abstracts routing and pipeline management, exposing two primary extension points via the AIContextProvider base class:
ProvideAIContextAsync (Pre-Call): Executes before the request reaches the LLM. Full access to the current session, previous instructions, and pending messages. Ideal for memory injection, intent-based tool filtering, and constraint framing.StoreAIContextAsync (Post-Call): Executes after LLM generation but before response delivery. Used for response validation, fact extraction, logging, or state persistence.
Providers execute in registration order, enabling layered context modifications. Earlier provider outputs are visible to downstream providers, ensuring transparent, predictable pipeline behavior.
1. Memory Injection
For agents requiring user-specific context (e.g., a barista agent), ProvideAIContextAsync fetches preferences from storage and appends them to instructions. StoreAIContextAsync runs a lightweight extractor on the user message to persist new facts for future sessions.
public class BaristaMemoryProvider : AIContextProvider
{
private const string UserIdStateKey = "UserId";
private readonly ICoffeeDatabase _db;
private readonly IExtractorAgent _extractor;
public BaristaMemoryProvider(ICoffeeDatabase db, IExtractorAgent extractor)
{
_db = db;
_extractor = extractor;
}
protected override async ValueTask<AIContext> ProvideAIContextAsync(
AIContextProvider.InvokingContext context,
CancellationToken cancellationToken = default)
{
string userId = GetUserId(context.Session);
var userPrefs = await _db.GetPreferencesAsync(userId, cancellationToken);
if (userPrefs is null)
{
return new AIContext();
}
return new AIContext
{
Instructions =
$"User Coffee Profile: Brewer: {userPrefs.Brewer}, " +
$"Ratio: {userPrefs.Ratio}, Roast: {userPrefs.RoastType}."
};
}
protected override async ValueTask StoreAIContextAsync(
AIContextProvider.InvokedContext context,
CancellationToken cancellationToken = default)
{
var lastUserMessage = context.RequestMessages
.LastOrDefault(m => m.Role == ChatRole.User)?
.Text;
if (string.IsNullOrWhiteSpace(lastUserMessage))
{
return;
}
var extractedFact = await _extractor.ExtractNewFactsAsync(lastUserMessage, cancellationToken);
if (extractedFact is not null)
{
string userId = GetUserId(context.Session);
await _db.SaveNewPreferenceAsync(userId, extractedFact, cancellationToken);
}
}
private static string GetUserId(AgentSession? session) =>
session?.StateBag.TryGetValue<string>(UserIdStateKey, out var userId) == true
? userId
: "anonymous";
}
When agents expose dozens of tools, sending full schemas per request is inefficient. A routing provider evaluates intent via a fast auxiliary agent and injects only the relevant tool subset.
public class GuitarTechToolProvider : AIContextProvider
{
private readonly IRoadieAgent _roadieRouter;
private readonly IToolRegistry _tools;
public GuitarTechToolProvider(IRoadieAgent roadieRouter, IToolRegistry tools)
{
_roadieRouter = roadieRouter;
_tools = tools;
}
protected override async ValueTask<AIContext> ProvideAIContextAsync(
AIContextProvider.InvokingContext context,
CancellationToken cancellationToken = default)
{
var lastMsg = context.RequestMessages
.LastOrDefault(m => m.Role == ChatRole.User)?
.Text;
var intent = await _roadieRouter.DetermineIntentAsync(lastMsg, cancellationToken);
var selectedTools = new List<AITool>();
switch (intent)
{
case Intent.ToneAndGear:
selectedTools.Add(_tools.GetTool("AmpEQDialer"));
selectedTools.Add(_tools.GetTool("PedalBoardRouter"));
break;
case Intent.MusicTheory:
selectedTools.Add(_tools.GetTool("ScaleGenerator"));
break;
}
return new AIContext
{
Tools = selectedTools
};
}
}
3. Guardrails & Validation
LLMs often ignore implicit constraints. A guardrail provider injects explicit boundary conditions pre-call, then runs a lightweight validator post-call to catch policy violations (e.g., suggesting adhesives for Lego builds). Violations can trigger logging, content stripping, or silent retries.
public class LegoGuardrailProvider : AIContextProvider
{
private readonly IValidatorAgent _validator;
public LegoGuardrailProvider(IValidatorAgent validator)
{
Pitfall Guide
- Ignoring Provider Execution Order:
AIContextProvider chains execute sequentially based on registration. If a downstream provider depends on context injected by an upstream one, misordering will cause missing variables or overridden instructions. Always map dependencies explicitly and register in strict dependency order.
- Context Pollution & Prompt Bleed: Appending unstructured or excessive data to
Instructions can confuse the LLM's attention mechanism or trigger context window limits. Use strict delimiters, JSON/schema formatting, and character limits. Validate injected context length before attachment.
- Blocking the Pipeline in Post-Call:
StoreAIContextAsync executes synchronously with the response lifecycle. Heavy DB writes, slow external API calls, or synchronous validation will stall user delivery. Offload non-critical persistence to background queues or use Task.Run/fire-and-forget patterns where eventual consistency is acceptable.
- State Management Blind Spots: Relying on
Session.StateBag without proper key isolation, type safety, or fallbacks leads to cross-user data leakage or null reference exceptions. Always validate state extraction, use strongly-typed keys, and implement graceful degradation (e.g., "anonymous" fallbacks) for unauthenticated sessions.
- Over-Filtering Tool Schemas: Aggressively stripping tools based on intent classification can cause failures when user requests span multiple domains or contain ambiguous phrasing. Implement confidence thresholds, fallback toolsets, or multi-intent routing to prevent capability gaps.
- Validator False Positives/Negatives: Lightweight guardrail agents using keyword matching often misclassify edge cases or domain-specific terminology. Pair lexical filters with semantic validation, maintain a suppression allowlist, and define clear retry/strip strategies to avoid breaking valid workflows.
Deliverables
- 📐 AIContextProvider Chaining Blueprint: Architecture diagram detailing pre-call/post-call pipeline flow, provider dependency mapping, and state isolation patterns. Includes registration sequences for memory, routing, and validation providers.
- ✅ Production Readiness Checklist:
- ⚙️ Configuration Templates: DI registration snippets for chaining multiple
AIContextProvider implementations, including intent router setup, DB extractor wiring, and validator agent injection. Ready for copy-paste into Program.cs or Startup.cs.
