How to use model router for Microsoft Foundry

Learn how to use the model router in Azure OpenAI to select the best model for your task.

Model router is a trained language model that selects the best large language model (LLM) to respond to a prompt in real time. It uses different preexisting models to deliver high performance and save on compute costs, all in one model deployment. To learn more about how model router works, its advantages, and limitations, see the Model router concepts guide. To understand the architecture and routing logic, see How model router works.

Supported models

You don’t need to separately deploy the supported large language models for use with model router, except for the Claude models. To use model router with your Claude models, first deploy them from the model catalog. Model router invokes the deployments if you select them for routing.

Model router version 2025-11-18 (latest)

Format	Model	Version
OpenAI	gpt-5.6-sol	2026-07-09
OpenAI	gpt-5.6-terra	2026-07-09
OpenAI	gpt-5.6-luna	2026-07-09
OpenAI	gpt-5.5	2026-04-24
OpenAI	gpt-5.4	2026-03-05
OpenAI	gpt-5.4-mini	2026-03-17
OpenAI	gpt-5.4-nano	2026-03-17
OpenAI	gpt-5.2	2025-12-11
OpenAI	gpt-5	2025-08-07
OpenAI	gpt-5-mini	2025-08-07
OpenAI	gpt-5-nano	2025-08-07
OpenAI	o4-mini	2025-04-16
OpenAI	gpt-4.1	2025-04-14
OpenAI	gpt-4.1-mini	2025-04-14
OpenAI	gpt-4.1-nano	2025-04-14
OpenAI	gpt-4o	2024-11-20
OpenAI	gpt-4o-mini	2024-07-18
OpenAI	gpt-oss-120b	1
Anthropic	claude-opus-4-8	1
Anthropic	claude-opus-4-7	1
Anthropic	claude-opus-4-6	1
Anthropic	claude-sonnet-4-5	20250929
Anthropic	claude-haiku-4-5	20251001
xAI	grok-4-1-fast-reasoning	1
xAI	grok-4	1
DeepSeek	DeepSeek-V3.2	1
Meta	Llama-4-Maverick-17B-128E-Instruct-FP8	1

Deploy a model router model

Model router is packaged as a single Foundry model that you deploy. Start by following the steps in the resource deployment guide.

To deploy programmatically without the portal, use the REST API examples in the deployment sections that follow.

If your organization uses the built-in Azure Policy for model deployment, make sure the policy’s allowed publishers include Microsoft (the publisher of model router) and the publisher of each model you deploy for routing (for example, Anthropic for Claude models). Otherwise, the policy blocks the deployment.

By default, model router deploys with the Balanced routing mode and routes across the full supported model set. You only need to change the routing mode or select a model subset when you want custom routing behavior.

Default deployment

Go to the Microsoft Foundry portal and navigate to the model catalog. Find model-router in the Models list and select it. Choose Default settings for the Balanced routing mode and route between all supported models.

The REST API deployment path targets the Microsoft Foundry account resource directly and doesn’t require a Foundry project. This makes it a good option for existing customers who deploy and manage Foundry models without a project association.

Before you run the REST examples, sign in with Azure CLI and save a management-plane bearer token as AZURE_AI_AUTH_TOKEN.

export AZURE_AI_AUTH_TOKEN=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv)

Deploy model router programmatically with the Azure Management REST API. The following example creates a default deployment and relies on the built-in Balanced routing mode and full supported model set.

curl -X PUT "https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.CognitiveServices/accounts/my-foundry-account/deployments/model-router-deployment?api-version=2025-10-01-preview" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $AZURE_AI_AUTH_TOKEN" \
    -d '{
        "sku": {"name": "GlobalStandard", "capacity": 10},
        "properties": {
                "model": {"format": "OpenAI", "name": "model-router", "version": "2025-11-18"}
        }
}'

Optional: customize deployment settings

To enable more configuration options, choose Custom settings.

Your deployment settings apply to all underlying chat models that model router uses.

• Don’t deploy the underlying chat models separately. Model router works independently of your other deployed models.
• Select a content filter when you deploy the model router model or apply a filter later. The content filter applies to all content passed to and from the model router; don’t set content filters for each underlying chat model.
• Your tokens-per-minute rate limit setting applies to all activity to and from the model router; don’t set rate limits for each underlying chat model.

Optional: change the routing mode

Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

Use the Routing mode dropdown to select a routing profile. This sets the routing logic for your deployment.

When to use each mode:

• Balanced (default): Most workloads. Optimizes cost while maintaining quality.
• Quality: Critical tasks like legal review, medical summaries, or complex reasoning.
• Cost: High-volume, budget-sensitive workloads like content classification or simple Q&A.

Changes to the routing mode can take up to five minutes to take effect.

Optional: route to a model subset

Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

The latest version of model router supports custom subsets: you can specify which underlying models to include in routing decisions. This gives you more control over cost, compliance, and performance characteristics.

In the model router deployment pane, select Route to a subset of models. Then select the underlying models you want to enable. You must select at least one model for routing. If no models are selected, the deployment uses the default model set for your routing mode.

New models introduced later are excluded by default until explicitly added.

To include models by Anthropic (Claude) in your model router deployment, you need to deploy them yourself to your Foundry resource. See Deploy and use Claude models.

Changes to the model subset can take up to five minutes to take effect.

Configure custom settings with the REST API

Use the following example when you want to set both the routing mode and a model subset in the same deployment request.

Add a routing block only when you want to override the default Balanced mode or restrict the routed model set. The following example keeps the combined custom request with both a routing mode and a model subset.

The deployment request body uses format, name, and version for the model router itself and for each model in the routing subset. Find the correct values for each model in the supported models table in this article.

curl -X PUT "https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.CognitiveServices/accounts/my-foundry-account/deployments/model-router-deployment?api-version=2025-10-01-preview" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AZURE_AI_AUTH_TOKEN" \
  -d '{
    "sku": {"name": "GlobalStandard", "capacity": 10},
    "properties": {
        "model": {"format": "OpenAI", "name": "model-router", "version": "2025-11-18"},
        "routing": {
            "mode": "balanced",
            "models": [
                {"format": "OpenAI", "name": "gpt-4.1", "version": "2025-04-14"},
                {"format": "OpenAI", "name": "gpt-5.6-sol", "version": "2026-07-09"},
                {"format": "Meta", "name": "Llama-4-Maverick-17B-128E-Instruct-FP8", "version": "1"}
            ]
        }
    }
}'

If you include Anthropic Claude models in the routing.models array, you must first deploy them to the same Foundry account with a matching SKU. Otherwise the request fails with an InvalidResourceProperties error. Deploy Claude models from the Foundry model catalog before you reference them in a model router deployment. See Deploy and use Claude models.

Test model router with Foundry Responses and Chat Completions

Call model router the same way you call any OpenAI chat model. Set the model parameter to the name of your model router deployment. You can use the Microsoft Foundry SDK with the Responses API or the OpenAI SDK with the Chat Completions API, in either Python or JavaScript/TypeScript.

Install the required packages before you run the samples:

• Foundry Responses (Python): pip install azure-ai-projects>=2.0.0 azure-identity
• Foundry Responses (JavaScript/TypeScript): npm install @azure/ai-projects @azure/identity
• Chat Completions (Python): pip install openai>=1.75.0
• Chat Completions (JavaScript/TypeScript): npm install openai @azure/identity

Foundry Responses

Python

"""
Foundry Model Router - Foundry Responses SDK (AIProjectClient) Example

This example demonstrates how to use the Azure AI Projects SDK
(AIProjectClient) to get an authenticated OpenAI client and call the
Responses API with a Foundry Model Router deployment.

NOTE: AIProjectClient requires Entra ID authentication (DefaultAzureCredential),
      not API keys. You must be logged in via `az login` before running this.

Prerequisites:
  - An Azure AI Foundry project with a "model-router" deployment
  - Azure CLI installed and logged in (`az login`)
  - A .env file in the repo root with AZURE_AI_PROJECT_ENDPOINT and
    MODEL_DEPLOYMENT_NAME

Usage:
  pip install -r requirements.txt
  az login
  python model-router-foundry-responses-sdk.py
"""

import os
from pathlib import Path

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential
from dotenv import load_dotenv

# Load environment variables from .env in the repo root
load_dotenv(Path(__file__).resolve().parent.parent / ".env", override=True)

project_endpoint = os.environ["AZURE_AI_PROJECT_ENDPOINT"]
deployment = os.environ["MODEL_DEPLOYMENT_NAME"]

# <foundry_responses>
with (
    DefaultAzureCredential() as credential,
    AIProjectClient(endpoint=project_endpoint, credential=credential) as project_client,
    project_client.get_openai_client() as openai_client,
):
    response = openai_client.responses.create(
        model=deployment,
        input="In one sentence, name the most popular tourist destination in Seattle.",
    )
# </foundry_responses>

    print("--- Foundry Responses SDK Output ---")
    print(f"Routed to model: {response.model}")
    print(f"Response:\n{response.output_text}")
    print(
        f"\nUsage: {response.usage.input_tokens} input + {response.usage.output_tokens} output = {response.usage.total_tokens} total tokens"
    )

JavaScript/TypeScript

import { DefaultAzureCredential } from "@azure/identity";
import { AIProjectClient } from "@azure/ai-projects";

const project = new AIProjectClient(
  process.env["FOUNDRY_PROJECT_ENDPOINT"]!,
  new DefaultAzureCredential(),
);

// Get an OpenAI-compatible client that works with all Foundry models
const client = project.getOpenAIClient();

const response = await client.responses.create({
  model: process.env["MODEL_ROUTER_DEPLOYMENT_NAME"] || "model-router",
  input: "Explain retrieval-augmented generation in one sentence.",
});

// The "model" field reveals which underlying model was selected
console.log(`Responded model: ${response.model}`);
console.log(response.output_text);

Chat Completions

Python

"""
Foundry Model Router - Chat Completions API Example

This example demonstrates how to use Azure OpenAI's Chat Completions API
with a Foundry Model Router deployment. Model Router automatically selects
the best underlying LLM for each prompt based on your routing mode
(Balanced, Quality, or Cost).

Prerequisites:
  - An Azure OpenAI resource with a "model-router" deployment
  - A .env file in the repo root with AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY,
    and MODEL_DEPLOYMENT_NAME

Usage:
  pip install -r ../requirements.txt
  python model-router-chat-completions.py
"""

import os
from pathlib import Path

from dotenv import load_dotenv
from openai import AzureOpenAI

# Load environment variables from .env in the repo root
load_dotenv(Path(__file__).resolve().parent.parent / ".env", override=True)

endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
api_key = os.environ["AZURE_OPENAI_API_KEY"]
deployment = os.environ["MODEL_DEPLOYMENT_NAME"]

# <chat_completion>
client = AzureOpenAI(
    azure_endpoint=endpoint,
    api_key=api_key,
    api_version="2024-10-21",
)

response = client.chat.completions.create(
    model=deployment,
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {
            "role": "user",
            "content": "In one sentence, name the most popular tourist destination in Seattle.",
        },
    ],
)
# </chat_completion>

print("--- Chat Completions Response ---")
print(f"Routed to model: {response.model}")
print(f"Response:\n{response.choices[0].message.content}")
print(
    f"\nUsage: {response.usage.prompt_tokens} prompt + {response.usage.completion_tokens} completion = {response.usage.total_tokens} total tokens"
)

JavaScript/TypeScript

import { AzureOpenAI } from "openai";
import {
  DefaultAzureCredential,
  getBearerTokenProvider,
} from "@azure/identity";

const endpoint = process.env["AZURE_OPENAI_ENDPOINT"]!;
const deploymentName =
  process.env["MODEL_ROUTER_DEPLOYMENT_NAME"] || "model-router";
const apiVersion = "2025-11-18";

const credential = new DefaultAzureCredential();
const azureADTokenProvider = getBearerTokenProvider(
  credential,
  "https://cognitiveservices.azure.com/.default",
);

const client = new AzureOpenAI({
  endpoint,
  azureADTokenProvider,
  apiVersion,
  deployment: deploymentName,
});

const completion = await client.chat.completions.create({
  model: deploymentName,
  messages: [
    {
      role: "user",
      content: "Explain retrieval-augmented generation in one sentence.",
    },
  ],
});

// The "model" field reveals which underlying model was selected
console.log(`Responded model: ${completion.model}`);
console.log(completion.choices[0].message.content);

For the full runnable samples, see Model Router samples in the foundry-samples repository.

• Reference: OpenAI Responses API (responses.create, both languages)
• Reference: OpenAI Chat Completions API (chat.completions.create, both languages)
• Reference: AIProjectClient (Python)
• Reference: AIProjectClient (JavaScript/TypeScript)
• Reference: AzureOpenAI (OpenAI Python SDK)
• Reference: AzureOpenAI (OpenAI JavaScript/TypeScript SDK)

Keep Chat Completions requests on the same model (preview)

The Chat Completions API is stateless, so your application sends the conversation history with each request. By default, model router evaluates each request independently and might select a different underlying model for a later turn. Session affinity lets your application identify related requests and asks model router to try the same eligible model first.

Session affinity can improve the opportunity for prompt-cache reuse when consecutive requests have overlapping prompt prefixes. It doesn’t inspect cache state or guarantee a cache hit.

Configure session affinity

After your application reads the endpoint, API key, and deployment name, create the client with the preview feature header. Then create an opaque, application-owned session ID that doesn’t contain secrets or personal information:

"""
Foundry Model Router - Chat Completions Session Affinity Example

This example demonstrates how to use one session ID across separate
Chat Completions requests so Model Router can keep using the associated
eligible model while preserving normal fallback behavior.

Prerequisites:
  - An Azure OpenAI resource with a "model-router" deployment
  - A .env file beside this script with AZURE_OPENAI_ENDPOINT,
    AZURE_OPENAI_API_KEY, and MODEL_DEPLOYMENT_NAME

Usage:
    pip install -r requirements.txt
    python model-router-chat-completions-session-affinity.py
"""

import os
import uuid
from pathlib import Path

from dotenv import load_dotenv
from openai import AzureOpenAI

load_dotenv(Path(__file__).resolve().parent / ".env", override=True)

endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
api_key = os.environ["AZURE_OPENAI_API_KEY"]
deployment = os.environ["MODEL_DEPLOYMENT_NAME"]

# <session_affinity_enable>
client = AzureOpenAI(
    azure_endpoint=endpoint,
    api_key=api_key,
    api_version="2024-10-21",
    default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)

# Use an opaque application-owned ID without secrets or personal information.
# A fresh value makes the first request in each run establish a new association.
session_id = f"trip-planner-{uuid.uuid4()}"
session_affinity = {
    "routing_config": {
        "session_affinity": {
            "mode": "sticky",
            "session_id": session_id,
        }
    }
}
# </session_affinity_enable>

messages = [
    {"role": "system", "content": "You are a helpful travel planner."},
    {"role": "user", "content": "Plan a one-day family trip to Seattle."},
]

# <session_affinity_turns>
first_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    extra_body=session_affinity,
)

messages.extend(
    [
        {"role": "assistant", "content": first_response.choices[0].message.content},
        {
            "role": "user",
            "content": "Add restaurant suggestions and indoor activities.",
        },
    ]
)

second_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    extra_body=session_affinity,
)
# </session_affinity_turns>

def print_response(turn, response):
    print(f"\n--- {turn} ---")
    print(f"Serving model: {response.model}")

    model_selection_details = getattr(response, "model_selection_details", None)
    if not model_selection_details:
        print("No model selection details were returned.")
    else:
        model_router_details = model_selection_details.get("model_router_details", {})

        # <session_affinity_extract>
        affinity_details = model_router_details.get("session_affinity")
        if not affinity_details:
            print("No session affinity details were returned.")
        else:
            print(f"Affinity mode: {affinity_details.get('mode', 'unknown')}")
            print(f"Affinity source: {affinity_details.get('source', 'unknown')}")
            print(f"Affinity decision: {affinity_details.get('decision', 'not reported')}")
        # </session_affinity_extract>

    print(f"Response:\n{response.choices[0].message.content}")

print_response("First turn", first_response)
print_response("Second turn", second_response)

Use the same session ID for every turn in one conversation. Use a different session ID for an unrelated conversation.

You can alternatively provide the application-owned identifier in the x-ms-session-id request header. When a request contains valid identifiers in both locations, routing_config.session_affinity.session_id takes precedence. A session ID must contain 1 through 256 Unicode code points, including at least one non-whitespace character. Model router ignores an invalid body identifier and tries a valid header identifier. If neither identifier is valid, Chat Completions uses normal routing.

Send related conversation turns

Send the first request, append its response and the next user message to the conversation history, and send the next request with the same session affinity configuration:

"""
Foundry Model Router - Chat Completions Session Affinity Example

This example demonstrates how to use one session ID across separate
Chat Completions requests so Model Router can keep using the associated
eligible model while preserving normal fallback behavior.

Prerequisites:
  - An Azure OpenAI resource with a "model-router" deployment
  - A .env file beside this script with AZURE_OPENAI_ENDPOINT,
    AZURE_OPENAI_API_KEY, and MODEL_DEPLOYMENT_NAME

Usage:
    pip install -r requirements.txt
    python model-router-chat-completions-session-affinity.py
"""

import os
import uuid
from pathlib import Path

from dotenv import load_dotenv
from openai import AzureOpenAI

load_dotenv(Path(__file__).resolve().parent / ".env", override=True)

endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
api_key = os.environ["AZURE_OPENAI_API_KEY"]
deployment = os.environ["MODEL_DEPLOYMENT_NAME"]

# <session_affinity_enable>
client = AzureOpenAI(
    azure_endpoint=endpoint,
    api_key=api_key,
    api_version="2024-10-21",
    default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)

# Use an opaque application-owned ID without secrets or personal information.
# A fresh value makes the first request in each run establish a new association.
session_id = f"trip-planner-{uuid.uuid4()}"
session_affinity = {
    "routing_config": {
        "session_affinity": {
            "mode": "sticky",
            "session_id": session_id,
        }
    }
}
# </session_affinity_enable>

messages = [
    {"role": "system", "content": "You are a helpful travel planner."},
    {"role": "user", "content": "Plan a one-day family trip to Seattle."},
]

# <session_affinity_turns>
first_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    extra_body=session_affinity,
)

messages.extend(
    [
        {"role": "assistant", "content": first_response.choices[0].message.content},
        {
            "role": "user",
            "content": "Add restaurant suggestions and indoor activities.",
        },
    ]
)

second_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    extra_body=session_affinity,
)
# </session_affinity_turns>

def print_response(turn, response):
    print(f"\n--- {turn} ---")
    print(f"Serving model: {response.model}")

    model_selection_details = getattr(response, "model_selection_details", None)
    if not model_selection_details:
        print("No model selection details were returned.")
    else:
        model_router_details = model_selection_details.get("model_router_details", {})

        # <session_affinity_extract>
        affinity_details = model_router_details.get("session_affinity")
        if not affinity_details:
            print("No session affinity details were returned.")
        else:
            print(f"Affinity mode: {affinity_details.get('mode', 'unknown')}")
            print(f"Affinity source: {affinity_details.get('source', 'unknown')}")
            print(f"Affinity decision: {affinity_details.get('decision', 'not reported')}")
        # </session_affinity_extract>

    print(f"Response:\n{response.choices[0].message.content}")

print_response("First turn", first_response)
print_response("Second turn", second_response)

Model router creates or updates a model association only after a successful request. The association expires after 30 minutes without a successful create or update.

Verify the affinity decision

Inspect model_selection_details.model_router_details.session_affinity to determine how model router applied affinity:

"""
Foundry Model Router - Chat Completions Session Affinity Example

This example demonstrates how to use one session ID across separate
Chat Completions requests so Model Router can keep using the associated
eligible model while preserving normal fallback behavior.

Prerequisites:
  - An Azure OpenAI resource with a "model-router" deployment
  - A .env file beside this script with AZURE_OPENAI_ENDPOINT,
    AZURE_OPENAI_API_KEY, and MODEL_DEPLOYMENT_NAME

Usage:
    pip install -r requirements.txt
    python model-router-chat-completions-session-affinity.py
"""

import os
import uuid
from pathlib import Path

from dotenv import load_dotenv
from openai import AzureOpenAI

load_dotenv(Path(__file__).resolve().parent / ".env", override=True)

endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
api_key = os.environ["AZURE_OPENAI_API_KEY"]
deployment = os.environ["MODEL_DEPLOYMENT_NAME"]

# <session_affinity_enable>
client = AzureOpenAI(
    azure_endpoint=endpoint,
    api_key=api_key,
    api_version="2024-10-21",
    default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)

# Use an opaque application-owned ID without secrets or personal information.
# A fresh value makes the first request in each run establish a new association.
session_id = f"trip-planner-{uuid.uuid4()}"
session_affinity = {
    "routing_config": {
        "session_affinity": {
            "mode": "sticky",
            "session_id": session_id,
        }
    }
}
# </session_affinity_enable>

messages = [
    {"role": "system", "content": "You are a helpful travel planner."},
    {"role": "user", "content": "Plan a one-day family trip to Seattle."},
]

# <session_affinity_turns>
first_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    extra_body=session_affinity,
)

messages.extend(
    [
        {"role": "assistant", "content": first_response.choices[0].message.content},
        {
            "role": "user",
            "content": "Add restaurant suggestions and indoor activities.",
        },
    ]
)

second_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    extra_body=session_affinity,
)
# </session_affinity_turns>

def print_response(turn, response):
    print(f"\n--- {turn} ---")
    print(f"Serving model: {response.model}")

    model_selection_details = getattr(response, "model_selection_details", None)
    if not model_selection_details:
        print("No model selection details were returned.")
    else:
        model_router_details = model_selection_details.get("model_router_details", {})

        # <session_affinity_extract>
        affinity_details = model_router_details.get("session_affinity")
        if not affinity_details:
            print("No session affinity details were returned.")
        else:
            print(f"Affinity mode: {affinity_details.get('mode', 'unknown')}")
            print(f"Affinity source: {affinity_details.get('source', 'unknown')}")
            print(f"Affinity decision: {affinity_details.get('decision', 'not reported')}")
        # </session_affinity_extract>

    print(f"Response:\n{response.choices[0].message.content}")

print_response("First turn", first_response)
print_response("Second turn", second_response)

A typical run produces output similar to the following example:

--- First turn ---
Serving model: <model-name>
Affinity mode: sticky
Affinity source: session_id_payload
Affinity decision: initialize
Response:
<first-response>

--- Second turn ---
Serving model: <model-name>
Affinity mode: sticky
Affinity source: session_id_payload
Affinity decision: retain
Response:
<second-response>

The first successful request typically returns initialize. A later request returns retain when the associated model serves the response. It returns switch when eligibility or fallback causes another model to serve the response. Policy, safety, capability, quota, availability, and fallback requirements take precedence over affinity.

To disable affinity for one request, set routing_config.session_affinity.mode to none. Model router uses normal routing for that request and doesn’t read or update the model association. The response reports mode as none and omits source and decision.

Session affinity is best-effort. An affinity lookup or persistence failure doesn’t fail inference. In that case, model router uses normal routing and omits the complete session_affinity response object. Session affinity doesn’t store conversation content, prevent fallback, or provide adaptive cache-aware switching.

For the response fields and fallback diagnostics, see Monitor model router.

Test model router in the playground

In the Foundry portal, go to your model router deployment on the Models + endpoints page and select it to open the model playground. In the playground, enter messages and see the model’s responses. Each response shows which underlying model the router selected.

You can set the Temperature and Top_P parameters to the values you prefer (see the concepts guide), but note that reasoning models (o-series) don’t support these parameters. If model router selects a reasoning model for your prompt, it ignores the Temperature and Top_P input parameters.

The parameters stop, presence_penalty, frequency_penalty, logit_bias, and logprobs are similarly dropped for o-series models but used otherwise.

Starting with the 2025-11-18 (latest) version, the reasoning_effort parameter (see the Reasoning models guide) is now supported in model router. If the model router selects a reasoning model for your prompt, it uses your reasoning_effort input value with the underlying model.

Connect model router to a Foundry agent

Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).

If you’ve created an AI agent in Foundry, you can connect your model router deployment to be used as the agent’s base model. Select it from the model dropdown menu in the agent playground. Your agent will have all the tools and instructions you’ve configured for it, but the underlying model that processes its responses will be selected by model router.

For detailed guidance on routing patterns, supported tool types, cost implications, and code examples for agents, see Use model router with Foundry agents.

For agentic requests, model router can select eligible OpenAI, open-source (OSS), and Anthropic models from your routing pool. Model and tool compatibility determine which models are eligible for each request. For current compatibility, see Tool support by region and model.

Output format

The standard Chat Completions response includes a "model" field that identifies the underlying model that served the request. You can also opt in to preview per-request metadata for routing attempts, status, and reported latency. For details, see Monitor model router.

The following example response was generated by using model router model version 2025-11-18:

{
    "success": true,
    "data": {
        "choices": [
            {
                "content_filter_results": {
                    "hate": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "protected_material_code": {
                        "filtered": false,
                        "detected": false
                    },
                    "protected_material_text": {
                        "filtered": false,
                        "detected": false
                    },
                    "self_harm": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "sexual": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "violence": {
                        "filtered": false,
                        "severity": "safe"
                    }
                },
                "finish_reason": "stop",
                "index": 0,
                "logprobs": null,
                "message": {
                    "annotations": [],
                    "content": "Charismatic and bold—combining brash showmanship and poetic wit with fierce competitiveness, moral conviction, and unwavering activism.",
                    "refusal": null,
                    "role": "assistant"
                }
            }
        ],
        "created": 1774543376,
        "id": "xxxx-yyyy-zzzz",
        "model": "gpt-5-mini-2025-08-07",
        "object": "chat.completion",
        "prompt_filter_results": [
            {
                "prompt_index": 0,
                "content_filter_results": {
                    "hate": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "jailbreak": {
                        "filtered": false,
                        "detected": false
                    },
                    "self_harm": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "sexual": {
                        "filtered": false,
                        "severity": "safe"
                    },
                    "violence": {
                        "filtered": false,
                        "severity": "safe"
                    }
                }
            }
        ],
        "system_fingerprint": null,
        "usage": {
            "completion_tokens": 163,
            "completion_tokens_details": {
                "accepted_prediction_tokens": 0,
                "audio_tokens": 0,
                "reasoning_tokens": 128,
                "rejected_prediction_tokens": 0
            },
            "prompt_tokens": 3254,
            "prompt_tokens_details": {
                "audio_tokens": 0,
                "cached_tokens": 3200
            },
            "total_tokens": 3417
        }
    }
}

Govern model router deployments with Azure Policy

If your organization restricts which models developers can deploy, model router honors the same built-in Foundry model deployment policy that governs standard model deployments. Policy is enforced at deploy time across the Foundry portal, REST API, Azure CLI, and ARM templates. For the IT admin assignment steps and the developer experience, see Govern model router deployments with Azure Policy.

Evaluate model router for your workload

Treat your initial deployment as a starting configuration. Before you send production traffic to model router, benchmark it against your current model for response quality, estimated cost, and latency. Use the results to decide whether to retain the configuration, change one routing lever, or keep a direct model deployment for part of the workload. For guidance, see Evaluate model router for your workload.

Monitor model router metrics

To inspect the serving model, routing attempts, status, and reported latency for an individual Chat Completions request, see Monitor model router.

Monitor performance

Monitor the performance of your model router deployment in Azure Monitor (AzMon) in the Azure portal.

1. Go to the Monitoring > Metrics page for your Azure OpenAI resource in the Azure portal.
2. Filter by the deployment name of your model router model.
3. Split the metrics by underlying models if needed.

Monitor costs

You can monitor the costs of model router, which is the sum of the costs incurred by the underlying models.

1. Visit the Resource Management -> Cost analysis page in the Azure portal.
2. If needed, filter by Azure resource.
3. Then, filter by deployment name: Filter by “Tag”, select Deployment as the type of the tag, and then select your model router deployment name as the value.

Troubleshoot model router

Common issues

Issue	Cause	Resolution
Rate limit exceeded	Too many requests to model router deployment	Increase tokens-per-minute quota or implement retry with exponential backoff
Unexpected model selection	Routing logic selected different model than expected	Review routing mode settings; consider using model subset to constrain options
High latency	Router overhead plus underlying model processing	Use Cost mode for latency-sensitive workloads; smaller models respond faster
Claude model not routing	Claude models require separate deployment	Deploy Claude models from model catalog before enabling in subset

Error codes

For API error codes and troubleshooting, see the Azure OpenAI REST API reference.

Resources

The following open-source repositories demonstrate model router in different scenarios. Each repo is on GitHub — learn, fork, and extend to accelerate your learning. Most samples require an existing model router deployment; see Deploy a model router model to get started.

Resource	Learn	Extend
Model Router Capabilities Interactive Demo (Python)	Compare Balanced, Cost, and Quality routing modes with custom prompts. View live benchmark data for cost savings, latency, and routing distribution.	Add your own prompt sets, integrate with your CI pipeline, or connect to your deployment for A/B testing.
Routed Models Distribution Analysis (Python)	Run batches of prompts across routing profiles and model subsets. See which models the router selects and in what proportions.	Plug in representative prompt logs to evaluate tradeoffs before adopting a routing policy at scale.
Multi-team scenarios with Quality & Cost benchmarking (Python, workshop)	Deploy model router, run benchmarks against fixed-model deployments, and analyze cost and latency optimization in a multi-team enterprise scenario.	Swap in your own models, prompts, and routing profiles to benchmark against your workload patterns.
On-Call Copilot Multi-Agent Demo (Python)	See how model router dynamically selects the right model per agent step — a fast, low-cost model for classification and a reasoning model for root-cause analysis.	Adapt the multi-agent architecture, agent roles, and escalation paths for your own operations or support scenarios.

These samples are intended for learning and experimentation only and are not production-ready. Before deploying any code derived from these repositories, review it against your organization’s security, compliance, and responsible AI policies. See the Microsoft Responsible AI principles for guidance.

Next steps

• Model router concepts - Learn how routing modes work
• Quotas and limits - Rate limits for model router
• Create an agent - Use model router with Foundry agents