Configure a voice agent

Configure the model, instructions, greeting, audio, and tools for a voice-based agent in Microsoft Foundry.

A voice-based agent in Microsoft Foundry Agent Service is an agent whose kind is voice. Its definition holds everything the service needs to run a spoken conversation: which model serves the session, what the agent says first, how it listens, how it sounds, and which tools it can call.

This article shows you how to set each part of that definition. Every create or update action produces a new immutable agent version, so you can change configuration and roll back without changing the agent’s endpoint. You can set these values with the Azure AI Projects client library, with azure.yaml and the Azure Developer CLI (azd), or visually in the Foundry portal playground. Select a configuration method to see the steps for that surface.

Items marked (preview) in this article are currently in public preview. This preview is provided without a service-level agreement, and we don’t recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Prerequisites

• A Foundry project with access to voice-based agents.
• Foundry User role on the project scope, or permission to create and update agents.

The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.

• A voice-based agent to configure. To create one, see Quickstart: Create a voice-based prompt agent.
• A voice model that’s available to your project. Use a service-managed model such as gpt-realtime, or your own Foundry model deployment. This corresponds to modelType: managed or, where supported, modelType: self_deployed with a bring-your-own-model (BYOM) deployment.
• A client that supports the audio input and output formats you configure.
• A microphone to test a live conversation in the playground.

Understand the voice agent definition

A voice agent definition includes these parts:

Part	Property	Purpose
Model	model_type, model	Selects which model serves the session.
Behavior	instructions, greeting, structured_inputs	Tells the agent what to do and what to say first.
Listening	audio.input	Sets input format, turn detection, noise reduction, and transcription.
Speaking	audio.output, output_modalities	Sets the voice, locale, speed, and what the agent emits.
Actions	tools, tool_choice, parallel_tool_calls	Defines what the agent can do during a call.
Latency	interim_response	Fills silence while the agent thinks or waits on a tool.
Data	store	Controls whether conversations are persisted.

The values in audio and avatar are session defaults. A client can override supported fields with a session.update event when it connects.

Related content

• Optimize voice agent instructions
• Best practices for voice-based agents
• Voice agent tracing, monitoring, and evaluation.
• Pricing for voice-based agents.
• Quickstart: Create a voice-based prompt agent
• Agent development with the Azure Developer CLI
• How to use the Voice Live API
• Publish and connect a voice-based agent