Azure SQL as a Knowledge Source in Foundry IQ: New Capabilities for AI Applications
Microsoft Foundry (formerly Azure ML) has added Azure SQL Database as a first-class knowledge source in Azure AI Search, available in public preview. This development enables enterprise developers and solution architects building Copilot, RAG, and agentic experiences to expose authoritative table data directly from Azure SQL Database, creating a structured ground-truth layer for large language models.
Why This Matters for RAG and Copilot
Traditional Retrieval-Augmented Generation (RAG) pipelines rely on unstructured document repositories — PDFs, wikis, HTML pages — where text is chunked and indexed without preserving relational structure. While effective for many use cases, this approach loses the semantic relationships, constraints, and referential integrity that make relational data valuable for enterprise AI applications.
The new Azure SQL knowledge source integration changes this dynamic by:
- Preserving schema structure: Table names, column definitions, data types, and primary/foreign key relationships are maintained during indexing, enabling more accurate retrieval and filtering.
- Supporting parameterized queries: When a user query triggers retrieval, the system can generate parameterized T-SQL queries against the Azure SQL database, ensuring consistent results and preventing injection vulnerabilities.
- The knowledge source connects live to the Azure SQL database, so Copilot responses reflect current data without requiring batch ETL pipelines.
- Maintaining security boundaries: Access control is inherited from the Azure SQL database’s existing permission model, including row-level security and column-level encryption.
Technical Implementation
To configure Azure SQL as a knowledge source in Foundry, administrators perform the following steps:
- Navigate to the Foundry workspace and open the AI Search section
- Create a new data connection to the target Azure SQL database using Microsoft Entra ID authentication
- Select specific tables or views to expose as searchable knowledge
- Configure column-level filtering to exclude sensitive data (PII, credentials, etc.)
- Define synonym maps and analyzers for natural language query optimization
- Test the connection and validate that queries return expected results
Example Use Cases
Copilot for Finance
Finance teams can connect Azure SQL databases containing general ledger data to Foundry, enabling natural language queries like “Show me the variance between Q2 budget and actual spending by department”. The relational structure ensures that budget vs. actual comparisons respect the chart-of-accounts hierarchy and fiscal period definitions.
Customer Service Copilot
Support organizations can index case resolution data from Azure SQL, allowing agents to ask “What was the resolution for case INC-4457 and what similar cases were resolved in the last 30 days?” The query leverages both the semantic search capabilities and the relational structure to find exact matches and related records.
Operations Monitoring
IoT and operations databases can provide real-time metrics and status information to operations Copilots: “Which devices have reported temperature alerts in the last hour and what are their current readings?” The live connection ensures real-time data without stale batch updates.
Limitations and Considerations
While powerful, the integration has important constraints:
- Public preview only: The feature is in public preview as of June 2026, meaning SLA guarantees and feature completeness may evolve.
- Query complexity: Very complex JOIN operations across multiple tables may not translate efficiently to AI Search’s retrieval model. Keep queries focused on specific tables or related sets.
- Data volume: Extremely large tables (millions of rows) may require indexed views or filtered partitions to maintain acceptable retrieval latency.
- Schema changes: Adding or modifying columns requires re-indexing to update the knowledge source. Plan schema changes with adequate lead time.
- Geographic restrictions: The Azure SQL database and Foundry workspace should be in the same region to minimize latency and comply with data residency requirements.
Getting Started Checklist
- Ensure you have an Azure SQL Database (Single Database, Elastic Pool, or Managed Instance)
- Verify Microsoft Entra ID authentication is configured for the database
- Identify which tables provide the highest-value knowledge for your AI use cases
- Create a test Foundry workspace or use an existing one
- Follow the official documentation to add Azure SQL as a knowledge source
- Start with a small subset of tables (2-3) to validate the workflow before broader deployment
Next Steps
As this feature moves from public preview to general availability, Microsoft is expected to add:
- Support for Azure SQL Managed Instances in VNet configurations
- Enhanced query guidance for complex multi-table JOIN scenarios
- Integration with Microsoft Fabric for combined lakehouse and relational workloads
- Expanded column-level security options for finer-grained data protection
For organizations building enterprise AI applications that require accurate, up-to-date relational data, Azure SQL as a knowledge source in Foundry IQ represents a significant step toward bridging the gap between structured database systems and unlarge language model reasoning capabilities.