Summary
Microsoft announced general availability of Foundry IQ in June 2026. It is a managed knowledge layer that lets developers ground AI agents in enterprise data without rebuilding retrieval pipelines for every project. Teams connect sources such as SharePoint, OneLake, and Azure Blob once, and Foundry IQ handles ingestion, chunking, vector indexing, and access enforcement using the calling user’s permissions (source). The knowledge base runs inside the same Foundry project as the agent, sharing its identity, observability, and content-safety configuration (source).
What this means
Foundry IQ shifts the operational burden of retrieval-augmented generation from custom code to a platform service. By unifying multiple data sources under a single agentic-retrieval endpoint backed by Azure AI Search (source), it eliminates the duplicated RAG plumbing that slows down multi-agent projects. The 36% response-quality lift over single-shot RAG reported by Microsoft (source) suggests the agentic retrieval engine’s iterative planning and source selection is more than marketing. For enterprises, the real win is keeping sensitive data inside the tenant boundary while still giving agents a unified, permission-aware view (source).
Actionable guidance for common pain points
1. Get the RBAC right before you deploy
The most frequent failure mode is a masked 401/405 error when Foundry Agent Service tries to call the knowledge-base MCP endpoint. The root cause is almost always missing role assignments on the Azure AI Search resource (source). Assign these roles to your Foundry project’s managed identity:
- Search Index Data Contributor – read/write access to index data
- Search Service Contributor – manage search service resources
If your knowledge base uses an LLM for query planning or answer synthesis, also grant Cognitive Services User to the Azure AI Search service’s managed identity on the Foundry resource (source).
2. Align project endpoints
A mismatched FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_PROJECT_RESOURCE_ID produces a “Connection not found” error. Verify both point to the same Foundry project before creating the MCP connection (source).
3. Use the base Azure OpenAI endpoint
Append /openai/v1 to the endpoint and you’ll get a 404 from the model endpoint. Use the base resource URL (e.g., https://<resource>.openai.azure.com) (source).
4. Isolate the failure by querying the knowledge base directly
Bypass Foundry Agent Service entirely to determine whether the problem is in Search or in the agent-to-Search integration. You can call the Retrieve REST API or the Python SDK KnowledgeBaseRetrievalClient with an admin key (source). If the direct query succeeds, the issue is in Foundry’s authentication/invocation path; if it fails with 401, re-check the RBAC roles above.
5. Choose the right retrieval reasoning effort
Foundry IQ exposes three reasoning-effort levels that control cost, latency, and depth of LLM-driven planning (source):
- Minimal – up to 10 sources, no LLM, no query planning.
- Low – up to 3 sources and 3 subqueries, answer synthesis with 5K-token budget.
- Medium – up to 5 sources and 5 subqueries, iterative search, 10K-token budget.
Start with Low for most production agents; bump to Medium only when you need multi-hop reasoning.
6. Prefer extractive output for agent consumption
Answer synthesis is required for web knowledge sources but adds latency and token cost. For agents that will reason over retrieved chunks, request extractive data instead (source).
7. Monitor with the chat playground and diagnostic logs
The Azure portal’s chat playground shows query plans, subqueries, and retrieval steps (source). Enable diagnostic logging on the search service for full request/response traces when you need to debug production issues.