Before You Let an AI Agent Use a Logged-In Browser, Define These 7 Boundaries

g ad-hoc or unverified operations.

// policy.types.ts

export interface IdentityContext {
  entity_id: string;
  profile_ref: string;
  region: string;
  locale: string;
  timezone: string;
}

export interface EnvironmentConstraints {
  proxy_id: string;
  allowed_exit_ips: string[];
  max_session_age_ms: number;
  strict_geo_match: boolean;
}

export interface ActionScope {
  allowed_domains: string[];
  permitted_actions: string[];
  blocked_actions: string[];
  requires_human_review: string[];
}

export interface AuditConfig {
  log_level: 'info' | 'debug' | 'trace';
  capture_screenshots: boolean;
  redact_secrets: boolean;
}

export interface SessionPolicy {
  id: string;
  identity: IdentityContext;
  constraints: EnvironmentConstraints;
  scope: ActionScope;
  audit: AuditConfig;
  created_at: string;
  version: number;
}

2. Implement the Governance Orchestrator

The orchestrator validates the environment and enforces constraints before delegating tasks to the agent. It ensures the browser profile matches the policy and that the network context is consistent.

// orchestrator.ts
import { BrowserContext, chromium } from 'playwright';
import { SessionPolicy } from './policy.types';

export class BrowserOrchestrator {
  private policy: SessionPolicy;
  private context: BrowserContext | null = null;

  constructor(policy: SessionPolicy) {
    this.policy = policy;
  }

  async initialize(): Promise<void> {
    // 1. Validate Profile Integrity
    await this.validateProfileIntegrity();

    // 2. Enforce Environment Constraints
    await this.enforceEnvironmentConstraints();

    // 3. Launch Context with Policy Bindings
    this.context = await this.launchBoundContext();
  }

  private async validateProfileIntegrity(): Promise<void> {
    // Check if the persistent profile exists and matches metadata
    // In production, this would query a profile registry
    const profileMeta = await this.getProfileMetadata(this.policy.identity.profile_ref);
    
    if (profileMeta.entity_id !== this.policy.identity.entity_id) {
      throw new Error(`PROFILE_MISMATCH: Profile ${this.policy.identity.profile_ref} belongs to ${profileMeta.entity_id}, expected ${this.policy.identity.entity_id}`);
    }
  }

  private async enforceEnvironmentConstraints(): Promise<void> {
    // Pre-flight check: Verify proxy exit IP matches expected region
    const exitIp = await this.resolveProxyExitIp(this.policy.constraints.proxy_id);
    
    if (this.policy.constraints.strict_geo_match) {
      const geo = await this.lookupGeo(exitIp);
      if (geo.country !== this.policy.identity.region) {
        throw new Error(`GEO_MISMATCH: Proxy exit IP ${exitIp} resolves to ${geo.country}, expected ${this.policy.identity.region}`);
      }
    }

    // Check session age
    const profileAge = Date.now() - this.policy.created_at;
    if (profileAge > this.policy.constraints.max_session_age_ms) {
      throw new Error(`SESSION_EXPIRED: Profile age exceeds maximum allowed duration.`);
    }
  }

  private async launchBoundContext(): Promise<BrowserContext> {
    // Launch with specific timezone, locale, and proxy
    const browser = await chromium.launchPersistentContext(
      this.policy.identity.profile_ref,
      {
        proxy: { server: `http://${this.policy.constraints.proxy_id}` },
        timezoneId: this.policy.identity.timezone,
        locale: this.policy.identity.locale,
        // Inject permissions based on scope
        permissions: this.mapPermissions(this.policy.scope),
      }
    );
    return browser;
  }

  // Helper methods for profile registry, proxy resolution, etc.
  // ...
}

3. Secure Credential Injection

Credentials must never be exposed to the agent's reasoning loop or logs. Use vault references that the orchestrator resolves internally. The agent receives a token or reference, not the secret.

// secrets.ts
import { SecretManagerClient } from '@google-cloud/secret-manager'; // Example provider

export class SecureCredentialResolver {
  private client: SecretManagerClient;

  constructor() {
    this.client = new SecretManagerClient();
  }

  async resolve(ref: string): Promise<string> {
    // Validate ref format to prevent injection
    if (!ref.startsWith('vault://')) {
      throw new Error('INVALID_REF: Credentials must use vault:// scheme.');
    }

    // Fetch secret from vault
    const [version] = await this.client.accessSecretVersion({ name: ref });
    const secret = version.payload?.data?.toString();
    
    if (!secret) {
      throw new Error('SECRET_NOT_FOUND');
    }

    return secret;
  }
}

// Usage in execution flow:
// const password = await resolver.resolve('vault://ent-9921/auth/main');
// await page.fill('#password', password);
// The LLM/Agent never sees the password value.

4. Rationale for Architecture Choices

• Decoupled Policy: By separating policy from code, you enable versioning, review, and reuse. Policies can be stored in GitOps repositories, allowing peer review of agent permissions.
• Pre-flight Validation: Checking proxy geo, profile integrity, and session age before launch prevents "drift" where an agent runs in an unintended context. This is cheaper and safer than detecting errors after execution.
• Vault References: Direct credential handling in automation scripts is a security anti-pattern. Vault references ensure secrets are rotated centrally and never leak into logs or agent memory.
• Scope Enforcement: Explicitly defining allowed and blocked actions prevents the agent from interpreting ambiguous UI elements as permission to perform destructive operations.

Pitfall Guide

Pitfall	Explanation	Fix
Profile Contamination	Reusing a persistent profile across multiple entities or tasks without isolation, leading to cross-account data leakage or state corruption.	Enforce strict profile-to-entity mapping. Implement profile reset or snapshot restoration between distinct entity runs.
Geo-Region Drift	The agent launches with a proxy that exits in a different region than the profile's expected context, triggering security flags or inconsistent UI.	Implement pre-flight IP geo-lookup. Fail fast if the proxy exit IP does not match the policy's region.
Credential Leakage	Logging passwords or API keys during execution, or passing them directly to the LLM prompt, exposing secrets in logs or model context.	Use vault references exclusively. Implement log redaction middleware. Never inject secrets into agent prompts.
Unbounded Autonomy	The agent interprets a generic instruction like "update settings" as permission to change security configurations or billing details.	Define granular permitted_actions and blocked_actions in the policy. Use human-review checkpoints for high-risk domains.
Silent Auth Failure	The agent assumes the session is valid based on the presence of cookies, but the session has expired or been revoked server-side.	Implement explicit auth-state verification steps (e.g., fetching user profile endpoint) before task execution.
Cache/IndexedDB Inconsistency	Relying only on cookies for session state, ignoring IndexedDB or local storage, causing the agent to miss critical state or UI elements.	Validate full browser context integrity. Ensure persistent profiles capture all storage mechanisms required by the target app.
Hardcoded Boundaries	Embedding permissions and constraints directly in the agent code, making them difficult to audit, update, or enforce consistently.	Externalize all boundaries into policy files. The agent should read and enforce policies, not define them.

Production Bundle

Action Checklist

• Define SessionPolicy Schema: Create a typed schema for identity, constraints, scope, and audit configuration.
• Implement Pre-flight Checks: Build validators for profile integrity, proxy geo-matching, and session age.
• Integrate Secret Vault: Replace all hardcoded credentials with vault references resolved by the orchestrator.
• Configure Audit Logging: Set up middleware to log entity_id, profile_ref, proxy_id, actions taken, and redacted secrets.
• Establish Human Review Triggers: Define conditions (e.g., payment pages, verification prompts) that pause execution for review.
• Test Profile Isolation: Verify that profiles cannot be swapped or contaminated across different entity runs.
• Deploy Policy Registry: Store policies in a version-controlled system with access controls.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
High-Frequency Monitoring	Governed Persistent Session	Maintains state for efficiency; governance ensures safety over repeated runs.	Medium (Infrastructure for profiles/vaults)
One-Off Sensitive Transaction	Ephemeral Session with Strict Scope	Minimizes attack surface; no persistent state to manage.	Low
Multi-Entity Dashboard Aggregation	Governed Persistent with Profile Rotation	Allows access to multiple accounts safely via isolated profiles and strict routing.	High (Complex orchestration)
Development/Testing	Ephemeral with Mock Data	Avoids risk to production data; faster iteration.	Low

Configuration Template

# session-policy.yaml
policy:
  id: "pol-ent-9921-monitoring-v1"
  identity:
    entity_id: "ent-9921"
    profile_ref: "profiles/ent-9921/main"
    region: "US"
    locale: "en-US"
    timezone: "America/New_York"
  constraints:
    proxy_id: "proxy-res-us-east-01"
    allowed_exit_ips:
      - "203.0.113.10"
      - "203.0.113.11"
    max_session_age_ms: 86400000 # 24 hours
    strict_geo_match: true
  scope:
    allowed_domains:
      - "dashboard.example.com"
      - "api.example.com"
    permitted_actions:
      - "read_dashboard"
      - "export_report"
      - "check_status"
    blocked_actions:
      - "modify_settings"
      - "initiate_payment"
      - "change_credentials"
    requires_human_review:
      - "verification_prompt"
      - "unexpected_login"
  audit:
    log_level: "info"
    capture_screenshots: true
    redact_secrets: true

Quick Start Guide

1. Initialize Project: Set up a TypeScript project with Playwright and your preferred secret manager SDK.
2. Define Policy: Create a session-policy.yaml file using the template above, tailored to your entity and requirements.
3. Build Orchestrator: Implement the BrowserOrchestrator class to load the policy, validate constraints, and launch the bound browser context.
4. Implement Agent Loop: Write your agent logic to execute tasks within the initialized context, respecting the ActionScope.
5. Run and Audit: Execute the agent and review the generated audit logs to verify identity, constraints, and actions. Iterate on policy as needed.