Quickstart: Optimize a hosted agent (preview)

Deploy and optimize a hosted agent by using the Azure Developer CLI, Python SDK, VS Code, or the Microsoft Foundry Skill.

Agent Optimizer is currently in preview. This preview is provided without a service-level agreement, and we don’t recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In this quickstart, you deploy the optimization sample agent, run the agent optimizer to improve its instructions, and deploy the winning candidate.

For the concepts behind each step and the full end-to-end path, see the optimization workflow.

Prerequisites

Before you begin, you need:

• An Azure subscription—Create one for free.

• Your Azure subscription must be on the allow list for the agent optimizer. Contact your Microsoft representative to request access.

The agent optimizer is currently in preview.

Clean up resources

If your workflow created resources through the AZD project, delete the provisioned resources when you finish experimenting:

azd down --force --purge

Why --purge? Foundry accounts use soft-delete by default. Without --purge, the resource name stays reserved for 48 hours, and reprovisioning with the same name fails.

Troubleshooting

Problem	Cause	Fix
azd ai agent optimize command not found	Extension too old	Run azd ext upgrade microsoft.foundry to get 0.1.40-preview or later.
optimization_model is required	Running in non-interactive mode without a model configured	Add --optimize-model gpt-5 to the command, or set optimization_model: gpt-5 under options: in eval.yaml. In interactive mode, the CLI prompts for model selection.
Python script fails with KeyError: 'DATASET_NAME' or another missing variable	The script didn’t load your .env file, or the variable is missing	Run the script from the same folder as .env, or export the required values in your shell before running python optimize_hosted_agent.py.
Python script fails with ResourceNotFound: The project does not exist	FOUNDRY_PROJECT_ENDPOINT doesn’t point to an existing Foundry project	Copy the project endpoint from the Foundry project’s Overview page and update FOUNDRY_PROJECT_ENDPOINT in .env.
Python script fails with Optimization model deployment '<name>' not found	OPTIMIZATION_MODEL is not the name of a deployed model in your Foundry project	Use the exact deployment name from Build > Deployments, such as an existing gpt-5 family or DeepSeek deployment in your project.
The job request fails with evaluators is required and cannot be empty	The Foundry-Features preview header is missing from the request	Add the AgentsOptimization=V2Preview header, as the FoundryFeaturesPolicy class in the C# path shows.
The job fails with No optimizable element found for the hosted agent	The request doesn’t include an optimizable target	Supply at least one target in optimization_config, such as the baseline system_prompt, the tool definitions, or skills.
The job fails with AllEvaluatorsFailedError	The evaluator is misconfigured, so every row fails to score	Open the evaluation run link in the error, and confirm the evaluator scores your agent’s responses. Start with a built-in evaluator such as builtin.task_adherence.
The Optimize section doesn’t appear for a hosted agent	Foundry Toolkit is older than version 1.6.4, or the selected agent isn’t a deployed hosted agent	Update Foundry Toolkit, reload Visual Studio Code, and reopen the deployed agent from the Agents tab.
GitHub Copilot Chat doesn’t open after you select the workspace	GitHub Copilot isn’t installed, isn’t available for your account, or agent mode is disabled	Set up GitHub Copilot in Visual Studio Code, enable agent mode, and then select New Optimization again.
Foundry Toolkit can’t apply the best candidate to the current workspace	The workspace doesn’t contain an azure.yaml service whose name matches the deployed hosted agent	Open the workspace that contains the selected agent’s code and matching azure.ai.agent service, then try again.
Coding agent can’t find the hosted agent	The wrong folder is open, or azure.yaml doesn’t define an azure.ai.agent service	Open the AZD project folder that contains azure.yaml, then ask the coding agent to resolve the hosted-agent service again.
Coding agent stops before applying or deploying a candidate	The Agent Optimizer skill requires review before source changes and deployment	Review the candidate scores and local diff, then explicitly approve the apply or deployment step.
Optimization score is 0 or very low	Evaluation has many errored rows	Open the Eval link in the results. Fix response generation or evaluator errors, then rerun.
azd provision fails with quota error	Subscription lacks capacity	Try a different region or request a quota increase.

What you learned

In this quickstart, you:

• Deployed the optimization sample agent by using the customer-support template.
• Ran the agent optimizer by using the Azure Developer CLI, Python SDK, .NET SDK, Visual Studio Code, or the Microsoft Foundry Skill.
• Deployed the winning candidate and verified the improvement.

Next steps

• Agent optimizer overview
• Make your agent optimizer-ready
• Create an evaluation dataset and evaluators
• Optimize agent instructions, skills, tools, and models
• Run agent evaluations with the azd CLI