Use admin-connected models in cloud evaluations

Learn how to use models connected through an enterprise AI gateway as evaluators, targets, and simulators in Microsoft Foundry cloud evaluations.

Admin-connected models are models hosted behind an enterprise AI gateway, such as Azure API Management or a non-Azure AI model gateway, that an administrator connects to your Foundry project. You can use an admin-connected model for cloud evaluation scenarios that accept a model deployment.

Foundry resolves the connection endpoint and authentication, including API key, managed identity, or OAuth 2.0 authentication. Your evaluation request references the connection and deployment, not the gateway endpoint or its credentials.

Cloud evaluation supports admin-connected models only when the connected deployment exposes the OpenAI Chat Completions API.

Admin-connected model support in cloud evaluation is in preview and might not be available in all regions.

Prerequisites

• A Foundry project.
• Foundry User role on the Foundry project.
• An administrator has created an Azure API Management or non-Azure AI model gateway connection on your Foundry resource and added the model on the Manage > Resource details > Admin-connected models tab. For setup instructions, see Bring your own model to Foundry Agent Service.
• The connection name and deployment name for a model that supports the OpenAI Chat Completions API.

Reference an admin-connected model

Use the following format anywhere a supported evaluation scenario accepts a model deployment:

<connection-name>/<deployment-name>

The following table shows common evaluation surfaces and the field that accepts the reference:

Scenario	Field
AI-assisted evaluator (judge)	initialization_parameters.model
Model target	target.model
Conversation simulation	item_generation_params.model

Use an admin-connected model as an evaluator judge model

Set initialization_parameters.model when you configure an AI-assisted evaluator. This example uses the admin-connected model as the judge for the coherence evaluator:

from azure.ai.projects.models import TestingCriterionAzureAIEvaluator

admin_connected_model = "my-apim-connection/gpt-4o"

testing_criteria = [
    TestingCriterionAzureAIEvaluator(
        type="azure_ai_evaluator",
        name="coherence",
        evaluator_name="builtin.coherence",
        initialization_parameters={"model": admin_connected_model},
        data_mapping={
            "query": "{{item.query}}",
            "response": "{{item.response}}",
        },
    ),
]

Use an admin-connected model as a target

Set target.model to send each evaluation input to the admin-connected model:

admin_connected_model = "my-apim-connection/gpt-4o"

target = {
    "type": "azure_ai_model",
    "model": admin_connected_model,
    "sampling_params": {
        "top_p": 1.0,
        "max_completion_tokens": 2048,
    },
}

Use this target with the model target evaluation flow described in Run evaluations in the cloud.

Use other evaluation scenarios

Other model-based scenarios in Run evaluations in the cloud by using the Microsoft Foundry SDK work similarly with admin-connected models. Wherever the scenario accepts a supported model deployment, replace the deployment name with <connection-name>/<deployment-name>. For example, conversation simulation accepts this reference in item_generation_params.model.

Keep the Foundry project endpoint unchanged, and don’t add the gateway endpoint or credentials to the evaluation request. Foundry resolves those values from the admin-connected model connection.

Admin-connected model isn’t available for synthetic data generation.

Related content

• Run evaluations in the cloud by using the Microsoft Foundry SDK
• Bring your own model to Foundry Agent Service