Monitor AI agent fleet health and performance - Microsoft Foundry

Track agent health, compliance, performance trends, and cost efficiency across your AI fleet by using Microsoft Foundry Control Plane monitoring.

Monitor agent health and performance across your fleet

As your organization scales from isolated copilots to autonomous multi-agent fleets, maintaining visibility and control becomes critical. Microsoft Foundry Control Plane provides a unified command center where you can monitor all agents, models, and tools across your enterprise from build to production.

Fleet monitoring serves multiple roles:

• Team managers gain oversight of agent operations and team productivity.
• Administrators enforce governance policies and track compliance posture.
• Cost managers optimize spending and identify resource inefficiencies.
• Security teams monitor for prohibited behaviors and policy violations.

This article shows you how to use Foundry Control Plane capabilities to track agent health, performance, compliance, and cost efficiency at scale. By using centralized monitoring, you can identify problems early, optimize resource consumption, and help ensure that your AI systems operate safely and reliably.

Prerequisites

• An Azure account with an active subscription. If you don’t have one, create a free Azure account, which includes a free trial subscription.

• A Foundry project. If you don’t have one, create a project.

• The following permissions:

  • Read access to the project and subscription that you want to view data for
  • Log Analytics Reader role or higher on the Application Insights resource that’s associated with your agent
  • Cost Management Reader role

This capability is available only in the Foundry (new) portal. Look for in the portal banner to confirm you’re using Foundry (new).

How monitoring works

Foundry Control Plane discovers all the agents that you can access. It uses the Application Insights resources that host your agents to help you monitor and diagnose those agents.

Foundry Control Plane supports:

Foundry Control Plane aggregates logs and metrics available across each Application Insights resource that’s connected to each agent.

Foundry Control Plane requires agents to log diagnostic information by following OpenTelemetry standards with semantic conventions for Generative AI applications. You don’t need to configure Application Insights on each resource, but doing so is strongly recommended. When Control Plane has this data, it can provide:

• Fleet health metrics: Track active agents, run completion rates, and error trends across your entire fleet.
• Cost and performance tracking: Monitor token usage, budget consumption, and resource efficiency across all agents.
• Anomaly detection: Identify cost spikes, performance degradation, and emerging issues through trend analysis.
• Drill-down analysis: Move from fleet-level metrics to individual agent traces and logs for detailed investigation.

Agents running on resources without Application Insights don’t have health metrics, cost tracking, or drill-down traces.

Configure monitoring

Follow these steps for each project where you want to configure monitoring:

1. On the toolbar, select Operate.

2. On the left pane, select Admin.

3. Under All projects, use the search box to look for your project.

4. Select the project.

5. Select the Connected resources tab.

6. Ensure that there’s an associated resource for the category AppInsights.

1. If there’s no associated resource, add one by selecting Add connection and then selecting Application Insights.

You can send traces to either different Application Insights resources or to the same resource, depending on your governance and security requirements.

Your project is now configured for observability and tracing. To verify that monitoring is active, go to Operate > Overview in the Foundry portal, select your project from the dropdown list, and confirm that agent metrics appear in the dashboard. Metrics might take a few minutes to populate after initial configuration.

View metrics

You can view aggregated metrics for all agents within a selected project by using Foundry. The Overview pane provides insights into fleet health, compliance, and performance trends.

1. Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

1. On the toolbar, select Operate.

2. The Overview pane displays common metrics and insights for all discovered agents within the subscription by default.

1. Use the project dropdown list to scope down the metrics to specific projects, if necessary.

2. Configure the date range by using the date selectors in the upper-right corner.

The Overview pane shows fleet-level health scores, alert summaries, active agent counts, error rates, and compliance metrics for the selected time range. Use this view to quickly assess the operational state of your fleet and identify areas that need attention.

View individual agent metrics

You can view all your assets under a specific project, along with top-level metrics, from Foundry.

1. Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

1. On the toolbar, select Operate.

2. On the left pane, select Assets.

3. Select the Agents tab.

The tab shows the details of agents discovered within the subscription. To learn about these details, see Agent inventory.

1. To view more granular information on the performance of an individual agent, select an agent. The pane that appears provides quick insights into the selected agent’s health and recent activity. Use it to identify problems and take corrective actions.

The pane contains these sections:

• Active alerts: View policy, security, and evaluation alerts grouped by severity and take action.
• Activity: See key metrics such as error rate over time, total run information, and information on token usage.

To learn more about how to manage individual agents, see Manage agents at scale.

Troubleshoot monitoring

If you don’t see expected metrics or agents in the dashboard, check the following common causes:

• Agents don’t appear in the Assets pane: Verify that you have read access to the subscription and project where the agents are deployed. Different users see different agents depending on their access level.
• Metrics are empty or missing: Confirm that the Application Insights resource is connected to the project (see Configure monitoring). Metrics can take several minutes to populate after the initial connection.
• Health metrics or traces aren’t available for a specific agent: The agent might be running on a resource without Application Insights configured. Connect an Application Insights resource to the agent’s project to enable health data and drill-down traces.
• Cost data doesn’t appear: Verify that you have the Cost Management Reader role on the subscription.

Related content

• What is Microsoft Foundry Control Plane?
• Manage agents at scale
• Register and manage custom agents