Integrations/Browser Use

Browser Use + Veto

Browser automation agents can navigate anywhere, fill any form, and click any button. Veto adds the authorization layer that keeps them inside your boundaries -- URL whitelisting, credential protection, and human approval for sensitive actions.

Start freeRead the docs

What is Browser Use?

Browser Use is an open-source Python library that gives AI agents full control of a web browser via Playwright. Agents can navigate URLs, fill forms, click buttons, extract page content, and take screenshots autonomously. It uses a Controller pattern for custom actions and supports any LLM provider. With full browser access, the agent inherits every privilege of the browser session it runs in.

The real risks of browser agents

In December 2025, Gartner recommended CISOs block AI browser agent usage entirely. Researchers demonstrated one-click hijacks of browser agents via crafted URL parameters that exfiltrated emails and calendar data. The core problem: browser agents inherit the user's full session -- cookies, credentials, and access to every logged-in service.

Credential exfiltration

Agent fills a login form on a phishing site, or types credentials into a form field on a malicious page that's been injected via prompt injection.

Payment triggering

Agent clicks "Purchase" or "Confirm Order" buttons when it was only supposed to check prices. No human reviews the action before money leaves the account.

PII extraction

Agent navigates to HR portals, payroll systems, or customer databases and extracts personally identifiable information from page content.

Prompt injection via web pages

Malicious web content instructs the browser agent to perform actions the user never requested -- hidden text on a page can redirect the agent's behavior.

Quickstart

1. Install

pip install veto browser-use

2. Wrap Browser Use actions with authorization

Browser Use's Controller pattern lets you define custom actions. Wrap each action with veto.guard() to enforce policies before the browser acts.

agent.pypython
from browser_use import Agent, Controller
from browser_use.agent.views import ActionResult
from langchain_openai import ChatOpenAI
from veto import Veto

veto = Veto(api_key="veto_live_...")
controller = Controller()

@controller.action("Navigate to URL")
async def go_to_url(url: str) -> ActionResult:
    decision = await veto.guard(
        tool="go_to_url",
        arguments={"url": url},
    )
    if decision.decision == 'deny':
        return ActionResult(error=f"Blocked: {decision.reason}")
    return ActionResult(extracted_content=f"Navigating to {url}")

@controller.action("Fill form field")
async def input_text(selector: str, value: str) -> ActionResult:
    decision = await veto.guard(
        tool="input_text",
        arguments={"selector": selector, "value": value},
    )
    if decision.decision == 'deny':
        return ActionResult(error=f"Blocked: {decision.reason}")
    return ActionResult(extracted_content=f"Filled {selector}")

@controller.action("Click element")
async def click_element(selector: str) -> ActionResult:
    decision = await veto.guard(
        tool="click_element",
        arguments={"selector": selector},
    )
    if decision.decision == 'deny':
        return ActionResult(error=f"Blocked: {decision.reason}")
    if decision.decision == 'require_approval':
        return ActionResult(
            error=f"Requires approval: {decision.approval_id}"
        )
    return ActionResult(extracted_content=f"Clicked {selector}")

agent = Agent(
    task="Check order status on the company portal",
    llm=ChatOpenAI(model="gpt-5.4"),
    controller=controller,
    max_steps=25,
)

result = await agent.run()
print(result)

3. Define browser policies

veto/policies.yamlyaml
version: "1.0"
name: Browser Use agent policies

rules:
  - id: whitelist-domains
    tools: [go_to_url]
    action: deny
    conditions:
      - field: arguments.url
        operator: not_matches
        value: "^https://(.*\\.company\\.com|.*\\.google\\.com|.*\\.github\\.com)/.*"
    reason: "Navigation restricted to approved domains"

  - id: block-admin-paths
    tools: [go_to_url]
    action: deny
    conditions:
      - field: arguments.url
        operator: matches
        value: ".*/admin/.*|.*/internal/.*|.*/settings/.*"
    reason: "Admin and internal paths are off-limits"

  - id: block-password-fields
    tools: [input_text]
    action: deny
    conditions:
      - field: arguments.selector
        operator: matches
        value: ".*password.*|.*passwd.*|.*secret.*"
    reason: "Password fields cannot be filled by agents"

  - id: block-credit-card-fields
    tools: [input_text]
    action: deny
    conditions:
      - field: arguments.selector
        operator: matches
        value: ".*credit.*card.*|.*card.?number.*|.*cvv.*|.*cvc.*"
    reason: "Payment fields cannot be filled by agents"

  - id: approve-form-submissions
    tools: [click_element]
    action: require_approval
    conditions:
      - field: arguments.selector
        operator: matches
        value: ".*submit.*|.*purchase.*|.*checkout.*|.*pay.*|.*confirm.*"
    approval:
      timeout_minutes: 10
      notify: [ops@company.com]

  - id: block-delete-buttons
    tools: [click_element]
    action: deny
    conditions:
      - field: arguments.selector
        operator: matches
        value: ".*delete.*|.*remove.*|.*cancel-account.*"
    reason: "Destructive actions blocked"

Before and after

Without Veto
before.pypython
from browser_use import Agent, Controller
from langchain_openai import ChatOpenAI

controller = Controller()

agent = Agent(
    task="Book a flight from SFO to JFK for next Monday",
    llm=ChatOpenAI(model="gpt-5.4"),
    controller=controller,
)

result = await agent.run()

The agent has unrestricted browser access. It can navigate anywhere, fill any form, click any button. No policy enforcement, no audit trail.

With Veto
after.pypython
from browser_use import Agent, Controller
from browser_use.agent.views import ActionResult
from langchain_openai import ChatOpenAI
from veto import Veto

veto = Veto(api_key="veto_live_...")

controller = Controller()

@controller.action("Navigate to URL")
async def go_to_url(url: str) -> ActionResult:
    decision = await veto.guard(
        tool="go_to_url",
        arguments={"url": url},
    )

    if decision.decision == 'deny':
        return ActionResult(
            error=f"Navigation blocked: {decision.reason}"
        )

    if decision.decision == 'require_approval':
        return ActionResult(
            error=f"Navigation requires approval: {decision.approval_id}"
        )

    return ActionResult(extracted_content=f"Navigating to {url}")

@controller.action("Fill form field")
async def input_text(selector: str, value: str) -> ActionResult:
    decision = await veto.guard(
        tool="input_text",
        arguments={"selector": selector, "value": value},
    )

    if decision.decision == 'deny':
        return ActionResult(
            error=f"Input blocked: {decision.reason}"
        )

    return ActionResult(extracted_content=f"Filled {selector}")

@controller.action("Click element")
async def click_element(selector: str) -> ActionResult:
    decision = await veto.guard(
        tool="click_element",
        arguments={"selector": selector},
    )

    if decision.decision == 'deny':
        return ActionResult(
            error=f"Click blocked: {decision.reason}"
        )

    if decision.decision == 'require_approval':
        return ActionResult(
            error=f"Click requires approval: {decision.approval_id}"
        )

    return ActionResult(extracted_content=f"Clicked {selector}")

agent = Agent(
    task="Book a flight from SFO to JFK for next Monday",
    llm=ChatOpenAI(model="gpt-5.4"),
    controller=controller,
)

result = await agent.run()

What Veto controls

Browser actionRiskVeto policy
Navigate to URLPhishing, internal admin accessDomain whitelisting, path blocking
Fill form fieldCredential leakage, PII exposureSelector-based blocking for password/card fields
Click elementPayment triggering, account deletionRequire approval for submit/purchase/delete
Extract contentData exfiltration, scraping PIIBlock extraction from sensitive pages
Take screenshotCapturing confidential screensBlock screenshots of financial/HR pages

Defense in depth for browser agents

1

URL allowlisting

Restrict navigation to a set of approved domains. Block admin panels, internal tools, and any domain not on the whitelist. The agent cannot navigate to a URL that doesn't match your patterns.

2

Credential protection

Block form fills on any field matching password, credit card, SSN, or API key selectors. No exceptions, no overrides. The agent physically cannot type credentials into form fields.

3

Click approval

Route clicks on submit, purchase, delete, and confirm buttons to human approval. The browser pauses until a human reviews and approves the action. Payment buttons never fire without explicit human consent.

4

Audit everything

Every browser action is logged with the URL, selector, input value, and policy decision. When something goes wrong, you have a complete record of what the agent did and why each action was allowed or blocked.

Frequently asked questions

How does Veto prevent credential theft by browser agents?
Veto intercepts the input_text action before it reaches the browser. Policies block any form fill on selectors matching password, credit card, SSN, or API key patterns. The agent receives a denial response and cannot bypass this protection. Even if a malicious page uses a non-standard selector name, you can add custom patterns to your policy.
Can an agent still click a purchase button accidentally?
Not with Veto. Define a require_approval policy for click actions on selectors matching purchase, checkout, pay, or confirm patterns. The browser agent pauses and routes the action to a human for approval. The click only executes after explicit human consent via the Veto dashboard, Slack, or API.
How do I restrict which websites my agent can visit?
Create a URL allowlist policy that only permits navigation to approved domains. Use glob patterns like https://*.company.com/* to allow your properties while blocking everything else. Admin paths, internal tools, and unknown domains are denied by default.
Does Veto protect against prompt injection via web pages?
Veto doesn't prevent prompt injection itself (that's an LLM-level concern), but it limits the damage. Even if a malicious page injects instructions that trick the agent into performing harmful actions, Veto's policies still enforce URL restrictions, credential protection, and click approvals. The agent cannot execute actions that violate your policies regardless of what the LLM decides.
What's the performance impact on browser automation?
Negligible. Veto evaluates policies in under 10ms. Browser actions (navigation, rendering, form fills) take 100-2000ms each. The authorization check is invisible compared to actual browser operation time.

Related integrations

Playwright

Guardrails for Playwright and Playwright MCP

PydanticAI

Type-safe Python agent authorization

MCP

Model Context Protocol tool authorization

Python SDK

Native Python authorization SDK

Secure your browser agents before they click the wrong button.

Start freeView on GitHub