Automate browser tasks with Foundry agents

Automate web browsing tasks with the Browser Automation tool in Microsoft Foundry agents. Create isolated Playwright sessions for navigation and form filling.

Items marked preview in this article are currently in preview. This preview is provided without a service-level agreement, and Microsoft doesn’t recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Consider adding this tool using a toolbox. By using a toolbox, you are able to reuse the tool across agents and runtimes, as well as centralizing credential management, versioning, and policy enforcement through a managed MCP endpoint. See the toolbox quickstart.

Browser Automation Tool (BAT) enables scalable, reliable browser-based automation within Foundry agents. BAT is available as an MCP tool powered by Playwright workspaces as its headless browser infrastructure layer. It integrates seamlessly with modern agentic workflows while providing enterprise-grade security, observability, and extensibility.

Browser Automation Tool (BAT) provides a comprehensive platform for browser automation through:

• Playwright Workspaces (a Generally Available service) as the infrastructure layer
• Real-time debugging with Live View
• Take control for human-in-the-loop scenarios
• Support for private website browsing (Public preview)
• Built-in observability for reliability and optimization
• Flexible orchestration layers

Browser Automation Tool controls web browsers on your behalf and can take unintended actions. Testing and monitoring are recommended. See the Foundry Agent Service transparency note to learn more.

Use cases

Use Browser Automation Tool for browser-based workflows that require navigation, information extraction, form interaction, or validation across websites and business applications.

For example, a finance operations team can use an agent with Browser Automation Tool to retrieve invoice details from a vendor portal, validate the information against its internal system, and enter the required data into a legacy finance application. The agent can use the browser when these systems don’t provide APIs or when the workflow spans multiple browser-based applications.

Common scenarios

• Form filling and third-party integration: Complete forms and move information between browser-based applications when an API isn’t available.

• Web research and data extraction: Navigate websites, filter results, and collect or compare information across sources.

• Quality assurance and regression testing: Validate end-to-end user flows, reproduce reported issues, and investigate failures in a real browser session.

• Customer support and issue replication: Re-create a customer journey, gather context from a website, and help support teams investigate browser-based issues.

• Legacy-system automation: Automate repetitive interactions with internal or vendor portals that don’t provide modern APIs.

• Employee and business operations: Support supervised workflows such as employee-lifecycle tasks, invoice processing, and other browser-based operational processes.

How it works

The Browser Automation tool enables a Foundry agent to interact with websites through a managed browser provided by Playwright Workspaces. Playwright Workspaces provides the cloud-hosted browser infrastructure, while the Browser Automation tool exposes browser capabilities that the agent can use to complete a user’s task.

When a user gives the agent a browser-based task, the agent determines which browser operations are needed to make progress toward the requested goal. The agent can use the available browser automation tools to create and interact with a browser session, navigate to webpages, inspect page content, find and interact with elements, enter information, and complete multi-step browser workflows.

A typical browser automation flow works as follows:

1. The user provides a natural-language request, such as asking the agent to visit a website, find information, complete a form, or perform a sequence of browser interactions.

2. When browser interaction is required, the agent uses the Browser Automation tool connected to a Playwright Workspace. The workspace provides the managed, remote browser environment in which the task runs.

3. The browser automation service creates a browser session. The session is identified by a browser session ID that is used across subsequent browser operations so the agent can continue working in the same browser context.

4. Browser automation tools allow the agent to navigate pages and inspect the current page state. Based on the user’s goal and the information available from the page, the agent determines the next browser operation to perform.

5. The agent can perform browser interactions such as navigating to another page, finding and selecting page elements, entering text, or interacting with controls. The resulting page state becomes available to subsequent browser operations.

6. For multi-step workflows, the agent repeats this observe-and-act process, using the same browser session as it moves through the website. This approach allows the agent to adapt its next action to the current state of the page instead of relying on a predefined sequence of browser steps.

7. After the workflow finishes, the browser session is closed and its associated browser resources are released.

Because the browser runs remotely in Playwright Workspaces, the agent doesn’t need a locally installed browser or Playwright runtime to perform these interactions. Playwright Workspaces also provides capabilities for observing and troubleshooting browser sessions, including browser session data and Live View.

This architecture enables agents to perform workflows such as navigating websites, retrieving and validating information, completing forms, interacting with authenticated applications, and carrying out multi-step browser processes.

Choose your setup path

Choose the agent type and development approach before you configure Browser Automation Tool. All options use a Playwright workspace and a Browser Automation toolbox. The difference is where your agent logic runs and how much code you manage.

Choose	Use this option when	Next step
Prompt agent	You want the fastest path to a browser-enabled agent. Define instructions, a model, and tools in Foundry, and let Foundry run the agent.	Follow the Prompt agent sample.
Hosted agent with Microsoft Agent Framework	You need custom agent code but want a recommended starting framework and Foundry-managed hosting, scaling, identity, and observability.	Follow the Hosted agent quickstart. Select Agent Framework when you create the sample.
Hosted agent with your own framework	You already use a framework such as LangGraph, OpenAI Agents SDK, Semantic Kernel, or custom code, and need to retain your existing orchestration logic.	Create a Hosted agent, then connect to the Browser Automation toolbox by using your framework’s MCP client.

Prerequisites

Before you begin, make sure you have:

• An Azure subscription. Create one for free.
• Foundry User role on the Foundry project for day-to-day agent development and use.

The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.

• Foundry Project Manager role on the Foundry project if you create the project connection.
• Contributor role on the target resource group only while you create the Playwright workspace. This role is required for resource provisioning. Activate it just in time through Microsoft Entra Privileged Identity Management (PIM), and deactivate it after provisioning. Day-to-day agent developers and runtime users don’t need this role.
• A Foundry project with a configured endpoint.
• An AI model deployed in your project (for example, gpt-5.4). Confirm that both the model and project region support Browser Automation in Tool support by region and model.
• A Playwright workspace resource.
• A project connection set up for your Playwright workspace.

SDK requirements

For Python examples, install the required packages:

pip install "azure-ai-projects>=2.0.0"

The .NET SDK is currently in preview. For more information, see the quickstart.

Configuration

Get your project endpoint: Open your project in the Foundry portal, and copy the endpoint from the project overview page. The format is https://{account-name}.services.ai.azure.com/api/projects/{project-name}.

Connection ID format: Use /subscriptions/{{subscriptionID}}/resourceGroups/{{resourceGroupName}}/providers/Microsoft.CognitiveServices/accounts/{{foundryAccountName}}/projects/{{foundryProjectName}}/connections/{{foundryConnectionName}}. You can find this value on the tool’s details page after you connect the Browser Automation tool.

Usage support

The following table shows SDK and setup support.

Microsoft Foundry support	Python SDK	C# SDK	JavaScript SDK	Java SDK	REST API	Basic agent setup	Standard agent setup
✔️	✔️	✔️	✔️	✔️	✔️	✔️	✔️

Choose an authentication type

When you create the Browser Automation connection, select the authentication method that meets your organization’s security requirements.

Authentication type	Use when	Considerations
API key	You are prototyping or your organization permits service access tokens.	The key is stored in the Foundry project connection. Don’t place it in source code, prompts, or logs. Rotate and revoke it according to your organization’s policy.
Project managed identity	Multiple agents in the same Foundry project need to access the same Playwright workspace.	Use the Foundry project’s Microsoft Entra identity and grant it only the required permissions on the workspace. This identity is the recommended default for enterprise workloads when supported by the connection.
Agent identity	A deployed Hosted agent needs its own access boundary for browser automation or downstream resources.	A Hosted agent receives a dedicated Microsoft Entra identity when it is deployed. Assign permissions to that identity when the agent requires access that shouldn’t be shared with other agents in the project.

Grant Playwright workspace access

Browser Automation Tool uses the Foundry project managed identity to access the Playwright workspace. This identity is different from a hosted agent’s dedicated agent identity:

• Project managed identity: Project-wide identity used by Foundry platform services and Browser Automation connections.
• Agent identity: Per-agent identity created when you deploy a hosted agent. Use it for the agent’s runtime access to downstream resources.

Grant the project managed identity the minimum permissions required to use the Playwright workspace:

1. In the Azure portal, open the Playwright workspace.
2. Select Access control (IAM), and then select Add > Add role assignment.
3. Select a custom role that contains only the Playwright permissions required by the Foundry project identity. If a custom role isn’t available, assign Playwright Workspace Contributor at the Playwright workspace resource scope.
4. On the Members tab, select the Foundry project managed identity.
5. Select Review + assign.

Set up Browser Automation

Create a Playwright workspace

1. In the Azure portal, create a Playwright Workspace resource.
2. After the workspace is created, go to Settings > Access Management.
3. Confirm the Playwright Service Access Token authentication method is enabled.
4. Select Generate Token, enter a name (for example, foundry-connection), and choose an expiry period.
5. Copy the token immediately. You can’t view it again after closing the page.
6. Store the token only in the Foundry project connection. Don’t put it in source code, prompts, or application logs. Rotate it before it expires, and revoke it immediately if it is exposed.
7. On the workspace Overview page, copy the Browser endpoint (it starts with wss://).
8. Configure a custom role with only the Playwright permissions that the Foundry project identity requires. If a custom role isn’t available, assign Contributor only at the Playwright workspace resource scope. The service access token is stored in the project connection; the role assignment separately authorizes the project identity to access the workspace resource.

Connect the Browser Automation tool in Foundry

1. Go to the Foundry portal and select your project.
2. Select Build > Tools.
3. Select Create a toolbox.
4. Fill in the Name and Description for your toolbox.
5. Under Tools, click on Add
6. Select Browser Automation and click Add tool
7. Enter the required fields
   • Connection name: Unique name for your connection
   • Playwright Workspace: Select the Playwright Workspace resource.
   • Auth Type: Select the authentication type for your connection.
8. Select Connect.
9. Click on Publish to save the toolbox

After the toolbox is created, you can view the Project connection ID on the tool’s details page. Use this value as the browser automation connection ID in your code.

Add browser automation to a toolbox with the Azure Developer CLI

To add browser automation to a Toolbox, use the Azure Developer CLI to create a Playwrite workspace. This article assumes you already have a Playwright workspace resource. See the prerequisites section.

1. Create the Playwright Workspace connection.

azd ai connection create my-browser-conn \
  --kind PlaywrightWorkspace \
  --target wss://your-browser-endpoint.api.playwright.microsoft.com/playwrightworkspaces/browsers \
  --auth-type api-key \
  --key "<playwright-workspaces-access-token>"

--kind PlaywrightWorkspace requires exact PascalCase.

2. Define the toolbox (my-toolbox.yaml)

description: Browser Automation toolbox
tools:
  - type: browser_automation_preview
    project_connection_id: my-browser-conn

3. Create the toolbox

azd ai toolbox create my-toolbox --from-file my-toolbox.yaml

Browser Automation tool definitions

After you run a sample, verify the tool was called by using tracing in Microsoft Foundry. For guidance on validating tool invocation, see Best practices for using tools in Microsoft Foundry Agent Service. If you use streaming, you can also look for browser_automation_preview_call events.

• The .NET SDK is currently in preview. For more information, see the quickstart.

ProjectsAgentTool tool = new BrowserAutomationPreviewTool(
    new BrowserAutomationToolOptions(
        new BrowserAutomationToolConnectionParameters("<BROWSER_AUTOMATION_PROJECT_CONNECTION_ID>")
    )
);

const tools = [
  {
    type: "browser_automation_preview",
    name: "<OPTIONAL_TOOL_NAME>",
    description: "<Optional description for the model>",
    browser_automation_preview: {
      connection: {
          project_connection_id: "<BROWSER_AUTOMATION_PROJECT_CONNECTION_ID>"
      }
    }
  },
];

Monitor browser automation sessions

Browser Automation Tool provides observability features to help you validate, troubleshoot, and review browser-based agent runs.

• Live View: Provides a real-time view of an active browser session. Use it during development and testing to verify navigation, page state, and browser interactions as the agent runs.
• Screenshot recording: Captures screenshots during an active browser session, helping you verify page state and browser interactions as the agent runs.
• Take control: Supports human-in-the-loop workflows. Use it when a person needs to intervene in a browser session before the agent continues.
• For production workflows, review run summaries, traces, and diagnostic logs to investigate outcomes without continuously monitoring active sessions.

Troubleshooting

The agent doesn’t use the tool

• Confirm you created the agent with the Browser Automation tool enabled.
• In your request, require tool usage (for example, tool_choice="required").
• Use tracing in Microsoft Foundry to confirm whether a tool call occurred. For guidance, see Best practices for using tools in Microsoft Foundry Agent Service.

Connection or authorization errors

• Confirm the browser automation connection ID matches the Playwright workspace connection resource ID in your project.
• Confirm the project identity has access to the Playwright workspace resource.
• If you recently rotated the Playwright access token, update the Foundry project connection key.

Python SDK errors

• Workspace not found: Verify your project endpoint uses the correct format: https://{account-name}.services.ai.azure.com/api/projects/{project-name}. Don’t use the legacy Azure ML endpoint format.
• Unexpected keyword argument errors: Ensure you’re using the latest version of azure-ai-projects. Run pip install "azure-ai-projects>=2.0.0" --upgrade to update.
• Import errors: Install all required packages: pip install "azure-ai-projects>=2.0.0".

Requests time out

Browser automation can take longer than typical requests.

• Increase the client timeout (the C# sample sets a 5-minute timeout).
• Reduce the scope of your prompt (for example, fewer pages and fewer interactions).

Limitations

• Sign-in and sensitive actions: Use this tool only with sites you trust. Avoid pages that prompt for credentials, payments, or other sensitive actions.
• Page volatility: Web pages can change at any time. Your agent might fail if the page layout, labels, or navigation flows change. Build error handling into your workflows.
• Complex single-page applications: JavaScript-heavy SPAs with dynamic content might not render correctly.

Cost considerations

This tool uses a Playwright workspace resource to run browser sessions. Review the Playwright workspace documentation for pricing and usage details. For guidance on optimizing tool usage, see Best practices for using tools in Microsoft Foundry Agent Service.

Clean up

• Delete the agent version you created for testing.
• Revoke or rotate the Playwright access token if you no longer need it.
• Remove the project connection if it’s no longer required. For more information, see Add a connection in Microsoft Foundry.

Transparency note

Review the transparency note when using this tool. The Browser Automation tool is a tool that can perform real-world browser tasks through natural language prompts, enabling automated browsing activities without human intervention.

Review the responsible AI considerations when using this tool.

Related content

• Best practices for using tools in Microsoft Foundry Agent Service
• Computer use tool for agents
• Add a connection in Microsoft Foundry
• Quickstart: Create your first agent