Connect a Foundry IQ knowledge base to Foundry Agent Service

Items marked (preview) in this article are currently in public preview. This preview is provided without a service-level agreement, and we don’t recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In this article, you learn how to connect a knowledge base in Foundry IQ to an agent in Foundry Agent Service. The connection uses the Model Context Protocol (MCP) to facilitate tool calls. When invoked by the agent, the knowledge base orchestrates the following operations:

Plans and decomposes a user query into subqueries.
Processes the subqueries simultaneously using keyword, vector, or hybrid techniques.
Applies semantic reranking to identify the most relevant results.
Synthesizes the results into a unified response with source references.

The agent uses the response to ground its answers in enterprise data or web sources, ensuring factual accuracy and transparency through source attribution. For an end-to-end example of integrating Azure AI Search and Foundry Agent Service for knowledge retrieval, see the agentic-retrieval-pipeline-example Python sample on GitHub.

Usage support

Microsoft Foundry support	Python SDK	C# SDK	JavaScript SDK	Java SDK	REST API	Basic agent setup	Standard agent setup
✔️	✔️	-	-	-	✔️	✔️	✔️

Prerequisites

An Azure AI Search service with a knowledge base containing one or more knowledge sources.
A Microsoft Foundry project with an LLM deployment, such as gpt-4.1-mini.
Authentication and permissions on your search service and project.
The latest preview Python SDK or the 2025-11-01-preview REST API version.
```
pip install azure-ai-projects azure-identity requests
```

Authentication and permissions

We recommend role-based access control for production deployments. If roles aren’t feasible, skip this section and use key-based authentication instead.

Microsoft Foundry
Azure AI Search

On the parent resource of your project, you need the Azure AI User role to access model deployments and create agents. Owners automatically get this role when they create the resource. Other users need a specific role assignment. For more information, see Role-based access control in Foundry portal.
On the parent resource of your project, you need the Azure AI Project Manager role to create a project connection for MCP authentication and either Azure AI User or Azure AI Project Manager to use the MCP tool in agents.
On your project, create a system-assigned managed identity for interactions with Azure AI Search.

On your search service, assign the Search Index Data Reader role to your project’s managed identity for read-only access to search indexes.
If your agent needs to write documents to search indexes, also assign the Search Index Data Contributor role.
For indexed content with access control lists (ACLs), include permission metadata fields in your search index and pass user tokens via the x-ms-query-source-authorization header at query time to filter results based on the user’s identity. For more information, see Query-time ACL and RBAC enforcement.
For remote SharePoint knowledge sources, the x-ms-query-source-authorization header passes the user’s identity, enabling SharePoint to enforce document permissions at query time. Content isn’t indexed. Instead, SharePoint applies permissions directly via the Copilot Retrieval API. For more information, see Create a remote SharePoint knowledge source.

Required values

Use the following values in the code samples.

Value	Where to get it	Example
Project endpoint (`project_endpoint`)	Find it in your project details in the Microsoft Foundry portal.	`https://your-resource.services.ai.azure.com/api/projects/your-project`
Project resource ID (`project_resource_id`)	Copy the project ARM resource ID from Azure portal or use Azure CLI to query the resource ID.	`/subscriptions/.../resourceGroups/.../providers/Microsoft.MachineLearningServices/workspaces/.../projects/...`
Azure AI Search endpoint (`search_service_endpoint`)	Find it on your Azure AI Search service Overview page (the service URL) in the Azure portal.	`https://your-search-service.search.windows.net`
Knowledge base name (`knowledge_base_name`)	Use the knowledge base name you created in Azure AI Search.	`hr-policy-kb`
Project connection name (`project_connection_name`)	Choose a name for the project connection you create.	`my-kb-mcp-connection`
Agent name (`agent_name`)	Choose a name for the agent version you create.	`hr-assistant`
Model deployment name (`deployed_LLM`)	Find it in your Microsoft Foundry project model deployments.	`gpt-4.1-mini`

We recommend you store the project endpoint, search endpoint, and knowledge base name in a .env file for local development.

Create a project connection

Create a RemoteTool connection on your Microsoft Foundry project. This connection uses the project’s managed identity to target the MCP endpoint of the knowledge base, allowing the agent to securely communicate with Azure AI Search for retrieval operations.

The RemoteTool category and ProjectManagedIdentity authentication type are specific to Microsoft Foundry project connections.

Python
REST

import requests
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# Provide connection details
credential = DefaultAzureCredential()
project_resource_id = "{project_resource_id}" # e.g. /subscriptions/{subscription}/resourceGroups/{resource_group}/providers/Microsoft.MachineLearningServices/workspaces/{account_name}/projects/{project_name}
project_connection_name = "{project_connection_name}"
mcp_endpoint = "{search_service_endpoint}/knowledgebases/{knowledge_base_name}/mcp?api-version=2025-11-01-preview" # This endpoint enables the MCP connection between the agent and knowledge base

# Get bearer token for authentication
bearer_token_provider = get_bearer_token_provider(credential, "https://management.azure.com/.default")
headers = {
  "Authorization": f"Bearer {bearer_token_provider()}",
}

# Create project connection
response = requests.put(
  f"https://management.azure.com{project_resource_id}/connections/{project_connection_name}?api-version=2025-10-01-preview",
  headers = headers,
  json = {
    "name": project_connection_name,
    "type": "Microsoft.MachineLearningServices/workspaces/connections",
    "properties": {
      "authType": "ProjectManagedIdentity",
      "category": "RemoteTool",
      "target": mcp_endpoint,
      "isSharedToAll": True,
      "audience": "https://search.azure.com/",
      "metadata": { "ApiType": "Azure" }
    }
  }
)

response.raise_for_status()
print(f"Connection '{project_connection_name}' created or updated successfully.")

Use the Azure CLI to get an access token for Azure Resource Manager:

az account get-access-token --scope https://management.azure.com/.default --query accessToken -o tsv

Create the project connection by making a PUT request to Azure Resource Manager:

PUT https://management.azure.com/{project_resource_id}/connections/{project_connection_name}?api-version=2025-10-01-preview
Authorization: Bearer {management_access_token}
Content-Type: application/json

{
  "name": "{project_connection_name}",
  "type": "Microsoft.MachineLearningServices/workspaces/connections",
  "properties": {
    "authType": "ProjectManagedIdentity",
    "category": "RemoteTool",
    "target": "{search_service_endpoint}/knowledgebases/{knowledge_base_name}/mcp?api-version=2025-11-01-preview", // This endpoint enables the MCP connection between the agent and knowledge base
    "isSharedToAll": true,
    "audience": "https://search.azure.com/",
    "metadata": {
      "ApiType": "Azure"
    }
  }
}

Optimize agent instructions for knowledge retrieval

To improve knowledge base invocations and produce citation-backed answers, start with instructions like the following:

You are a helpful assistant.

Use the knowledge base tool to answer user questions.
If the knowledge base doesn't contain the answer, respond with "I don't know".

When you use information from the knowledge base, include citations to the retrieved sources.

This instruction template optimizes for:

Higher MCP tool invocation rates: Explicit directives ensure the agent consistently calls the knowledge base tool rather than relying on its training data.
Clear source attribution: Citations make it easier to validate where information came from.

While this template provides a strong foundation, evaluate and iterate on the instructions based on your specific use case and objectives. Test different variations to find what works best for your scenario.

Create an agent with the MCP tool

Create an agent that integrates the knowledge base as an MCP tool. The agent uses a system prompt to instruct when and how to call the knowledge base. It follows instructions on how to answer questions and automatically maintains its tool configuration and settings across conversation sessions. Add the knowledge base MCP tool with the project connection you previously created. This tool orchestrates query planning, decomposition, and retrieval across configured knowledge sources. The agent uses this tool to answer queries.

Azure AI Search knowledge bases expose the knowledge_base_retrieve MCP tool for agent integration. This is the only tool currently supported for use with Foundry Agent Service.

Python
REST

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition, MCPTool
from azure.identity import DefaultAzureCredential

# Provide agent configuration details
credential = DefaultAzureCredential()
mcp_endpoint = "{search_service_endpoint}/knowledgebases/{knowledge_base_name}/mcp?api-version=2025-11-01-preview"
project_endpoint = "{project_endpoint}" # e.g. https://your-foundry-resource.services.ai.azure.com/api/projects/your-foundry-project
project_connection_name = "{project_connection_name}"
agent_name = "{agent_name}"
agent_model = "{deployed_LLM}" # e.g. gpt-4.1-mini

# Create project client
project_client = AIProjectClient(endpoint = project_endpoint, credential = credential)

# Define agent instructions (see "Optimize agent instructions" section for guidance)
instructions = """
You are a helpful assistant that must use the knowledge base to answer all the questions from user. You must never answer from your own knowledge under any circumstances.
Every answer must always provide annotations for using the MCP knowledge base tool and render them as: `【message_idx:search_idx†source_name】`
If you cannot find the answer in the provided knowledge base you must respond with "I don't know".
"""

# Create MCP tool with knowledge base connection
mcp_kb_tool = MCPTool(
    server_label = "knowledge-base",
    server_url = mcp_endpoint,
    require_approval = "never",
    allowed_tools = ["knowledge_base_retrieve"],
    project_connection_id = project_connection_name
)

# Create agent with MCP tool
agent = project_client.agents.create_version(
    agent_name = agent_name,
    definition = PromptAgentDefinition(
        model = agent_model,
        instructions = instructions,
        tools = [mcp_kb_tool]
    )
)

print(f"Agent '{agent_name}' created or updated successfully.")

Get an access token for Microsoft Foundry:

az account get-access-token --scope https://ai.azure.com/.default --query accessToken -o tsv

Create the agent by sending a POST request to Foundry Agent Service:

POST {project_endpoint}/agents/{agent_name}/versions?api-version=2025-11-15-preview
Authorization: Bearer {foundry_access_token}
Content-Type: application/json

{
  "definition": {
    "model": "{deployed_llm}",
    "instructions": "\nYou are a helpful assistant that must use the knowledge base to answer all the questions from user. You must never answer from your own knowledge under any circumstances.\nEvery answer must always provide annotations for using the MCP knowledge base tool and render them as: `【message_idx:search_idx†source_name】`\nIf you cannot find the answer in the provided knowledge base you must respond with \"I don't know\".\n",
    "tools": [
      {
        "server_label": "knowledge-base",
        "server_url": "{search_service_endpoint}/knowledgebases/{knowledge_base_name}/mcp?api-version=2025-11-01-preview",
        "require_approval": "never",
        "allowed_tools": [
          "knowledge_base_retrieve"
        ],
        "project_connection_id": "{project_connection_name}",
        "type": "mcp"
      }
    ],
    "kind": "prompt"
  }
}

Connect to a remote SharePoint knowledge source

In this preview, Foundry Agent Service doesn’t support per-request headers for MCP tools. Headers set in agent definitions apply to all invocations and can’t vary by user or request.For per-user authorization, use the Azure OpenAI Responses API instead.

Optionally, if your knowledge base includes a remote SharePoint knowledge source, you must also include the x-ms-query-source-authorization header in the MCP tool connection.

Python
REST

from azure.identity import get_bearer_token_provider

# Create MCP tool with SharePoint authorization header
mcp_kb_tool = MCPTool(
    server_label = "knowledge-base",
    server_url = mcp_endpoint,
    require_approval = "never",
    allowed_tools = ["knowledge_base_retrieve"],
    project_connection_id = project_connection_name,
    headers = {
        "x-ms-query-source-authorization": get_bearer_token_provider(credential, "https://search.azure.com/.default")()
    }
)

Get an access token for Azure AI Search:

az account get-access-token --scope https://search.azure.com/.default --query accessToken --output tsv

Provide the header and token in the MCP tool configuration:

    "tools": [
      {
        "server_label": "knowledge-base",
        "server_url": "{search_service_endpoint}/knowledgebases/{knowledge_base_name}/mcp?api-version=2025-11-01-preview",
        "require_approval": "never",
        "allowed_tools": [
          "knowledge_base_retrieve"
        ],
        "project_connection_id": "{project_connection_name}",
        "type": "mcp",
        "headers": {
            "x-ms-query-source-authorization": "{search-bearer-token}"
        }
      }
    ]

Invoke the agent with a query

Create a conversation session and send a user query to the agent. When appropriate, the agent orchestrates calls to the MCP tool to retrieve relevant content from the knowledge base. The agent then synthesizes this content into a natural-language response that cites the source documents.

Python
REST

# Get the OpenAI client for responses and conversations
openai_client = project_client.get_openai_client()

# Create conversation
conversation = openai_client.conversations.create()

# Send request to trigger the MCP tool
response = openai_client.responses.create(
    conversation = conversation.id,
    input = """
        Why do suburban belts display larger December brightening than urban cores even though absolute light levels are higher downtown?
        Why is the Phoenix nighttime street grid is so sharply visible from space, whereas large stretches of the interstate between midwestern cities remain comparatively dim?
    """,
    extra_body = {"agent": {"name": agent.name, "type": "agent_reference"}},
)

print(f"Response: {response.output_text}")

The output should be similar to the following (truncated for brevity):

Response: Suburban belts display larger December brightening than urban cores, even 
though absolute light levels are higher downtown, primarily because holiday lights 
increase most dramatically in the suburbs and outskirts of major cities. This is due 
to more yard space and a prevalence of single-family homes in suburban areas...

The Phoenix nighttime street grid is sharply visible from space due to the city's 
layout along a regular grid of city blocks and streets with extensive street lighting...

References:
- earth_at_night_508_page_174, earth_at_night_508_page_176 (Holiday lighting)
- earth_at_night_508_page_104, earth_at_night_508_page_105 (Phoenix grid visibility)

Send an empty POST request to create a conversation session:

### Create conversation
POST {project_endpoint}/openai/conversations?api-version=2025-11-15-preview
Authorization: Bearer {foundry_access_token}
Content-Type: application/json

{}

The response includes a conversation id, which you can use to send a query to the agent:

### Send request to trigger the MCP tool
POST {project_endpoint}/openai/responses?api-version=2025-11-15-preview
Authorization: Bearer {foundry_access_token}
Content-Type: application/json

{
    "conversation": "{conversation_id}",
    "input": "\nWhy do suburban belts display larger December brightening than urban cores even though absolute light levels are higher downtown?\nWhy is the Phoenix nighttime street grid is so sharply visible from space, whereas large stretches of the interstate between midwestern cities remain comparatively dim?\n",
    "agent": {
        "name": "{agent_name}",
        "type": "agent_reference"
    }
}

The response includes metadata about the agent execution, tool calls, and the generated output. The most relevant part of the response is the text in the content field, which should be similar to the following (truncated for brevity):

Suburban belts display larger December brightening in nighttime lights than urban 
cores, primarily because suburban areas have more yard space and single-family homes 
where holiday lighting decorations are more commonly used...

The Phoenix nighttime street grid is sharply visible due to the regular, planned 
layout of city blocks with extensive street lighting, shopping centers, and 
commercial properties along major streets...

References:
- earth_at_night_508_page_174_verbalized, earth_at_night_508_page_176_verbalized
- earth_at_night_508_page_104_verbalized, earth_at_night_508_page_105_verbalized

Delete the agent and project connection

    # Delete the agent
    project_client.agents.delete_version(agent.name, agent.version)
    print(f"Agent '{agent.name}' version '{agent.version}' deleted successfully.")

    # Delete the project connection (Azure Resource Manager)
    import requests
    from azure.identity import DefaultAzureCredential, get_bearer_token_provider

    credential = DefaultAzureCredential()
    project_resource_id = "{project_resource_id}"
    project_connection_name = "{project_connection_name}"

    bearer_token_provider = get_bearer_token_provider(credential, "https://management.azure.com/.default")
    headers = {"Authorization": f"Bearer {bearer_token_provider()}"}

    response = requests.delete(
      f"https://management.azure.com{project_resource_id}/connections/{project_connection_name}?api-version=2025-10-01-preview",
      headers=headers,
    )
    response.raise_for_status()
    print(f"Project connection '{project_connection_name}' deleted successfully.")

Deleting your agent and project connection doesn’t delete your knowledge base or its knowledge sources. You must delete these objects separately on your Azure AI Search service. For more information, see Delete a knowledge base and Delete a knowledge source.

Troubleshooting

This section helps you troubleshoot common issues when connecting Foundry Agent Service to a Foundry IQ knowledge base.

Authorization failures (401/403)

If you get a 403 from Azure AI Search, confirm the project’s managed identity has the Search Index Data Reader role on the search service (and Search Index Data Contributor if you write to indexes).
If you get a 403 from Azure Resource Manager when you create or delete the project connection, confirm your user or service principal has permissions on the Microsoft Foundry resource and project.
If you use keyless authentication, confirm your environment is signed in to the correct tenant and subscription.

MCP endpoint errors (400/404)

Confirm search_service_endpoint is the Azure AI Search service URL, such as https://<name>.search.windows.net.
Confirm knowledge_base_name matches the knowledge base you created in Azure AI Search.
Confirm you use the 2025-11-01-preview API version for the knowledge base MCP endpoint.

The agent doesn’t ground answers

Confirm the agent has the MCP tool configured and allowed_tools includes knowledge_base_retrieve.
Update your agent instructions to explicitly require using the knowledge base and to return “I don’t know” when retrieval doesn’t contain the answer.

What is Microsoft Foundry (new)?

Get started

Agent development

Agent tools & integration

Model capabilities

Fine-tuning

Manage agents, models, & tools

Observability, evaluation, & tracing

Developer experience

API & SDK

Responsible AI

Best practices

Setup & configure

Security & governance

Operate & support

Connect Agents to Foundry IQ Knowledge Bases

Connect a Foundry IQ knowledge base to Foundry Agent Service

Usage support

Prerequisites

Authentication and permissions

Required values

Create a project connection

Optimize agent instructions for knowledge retrieval

Create an agent with the MCP tool

Connect to a remote SharePoint knowledge source

Invoke the agent with a query

Delete the agent and project connection

Troubleshooting

Authorization failures (401/403)

MCP endpoint errors (400/404)

The agent doesn’t ground answers

What is Microsoft Foundry (new)?

Get started

Agent development

Agent tools & integration

Model capabilities

Fine-tuning

Manage agents, models, & tools

Observability, evaluation, & tracing

Developer experience

API & SDK

Responsible AI

Best practices

Setup & configure

Security & governance

Operate & support

​Connect a Foundry IQ knowledge base to Foundry Agent Service

​Usage support

​Prerequisites

​Authentication and permissions

​Required values

​Create a project connection

​Optimize agent instructions for knowledge retrieval

​Create an agent with the MCP tool

​Connect to a remote SharePoint knowledge source

​Invoke the agent with a query

​Delete the agent and project connection

​Troubleshooting

​Authorization failures (401/403)

​MCP endpoint errors (400/404)

​The agent doesn’t ground answers

​Related content

Connect a Foundry IQ knowledge base to Foundry Agent Service

Usage support

Prerequisites

Authentication and permissions

Required values

Create a project connection

Optimize agent instructions for knowledge retrieval

Create an agent with the MCP tool

Connect to a remote SharePoint knowledge source

Invoke the agent with a query

Delete the agent and project connection

Troubleshooting

Authorization failures (401/403)

MCP endpoint errors (400/404)

The agent doesn’t ground answers

Related content