Building a Practical Enterprise Data Agent with Microsoft Fabric Data Agent and Azure AI Foundry

Over the past year, I have been paying more attention to one question:

How do we move from a simple AI demo to a real enterprise AI application that can answer business questions using governed internal data?

A normal chatbot is easy to build. A useful enterprise AI agent is harder.

The hard part is not only choosing a large language model. The hard part is connecting the model to the right data, giving it the right business context, controlling what it can and cannot do, testing whether the answer is reliable, and then exposing that agent through an endpoint or SDK so it can be used inside real applications.

This is where Microsoft Fabric Data Agent and Azure AI Foundry, now documented through Microsoft Foundry and Foundry Agent Service, become interesting.

Microsoft Foundry is positioned as a unified Azure platform for enterprise AI operations, model builders, and application development. Foundry Agent Service is a managed platform for building, deploying, and scaling AI agents.

In this post, I will walk through how I think about the architecture, what each component does, and why this combination matters for data analysts, Power BI developers, and data scientists who want to build production-grade AI applications.

The Problem: Enterprise Data Is Not Ready for Chat by Default

When people first see an AI data agent, the demo usually looks very simple.

A user types:

Show sales by country

The agent responds:

At first glance, this looks like a basic natural language query. But in an enterprise environment, a lot has to happen behind the scenes before this answer can be trusted.

The agent needs to know:

• Which data source should I use?
• What does sales mean?
• Is it gross sales, net sales, booked sales, or recognized revenue?
• Which date field should be used?
• Which countries are valid?
• Should the answer include cancelled transactions?
• Does the user have permission to see this data?
• Should I return a direct number, a table, or a chart?
• What should I do if the question is ambiguous?

This is why enterprise AI is not just about the LLM.

The LLM is only one layer. A real solution also needs data source connections, a business semantic layer, agent instructions, security and permissions, a testing workflow, a publishing mechanism, SDK or API integration, monitoring, and governance.

This is the gap that Microsoft Fabric Data Agent and Azure AI Foundry are trying to fill.

High-Level Architecture

The architecture can be understood as two major layers.

Enterprise Data
    |
    v
Microsoft Fabric Data Agent
    |
    v
Published Agent / MCP Endpoint
    |
    v
Azure AI Foundry Agent
    |
    v
SDK / API / Business Application / Copilot / Internal Tool

In this structure, Microsoft Fabric Data Agent focuses on the data side. It connects to enterprise data sources and allows users to ask questions using natural language. Microsoft describes Fabric Data Agents as a way to create conversational AI experiences over data in Fabric.

Azure AI Foundry, or Microsoft Foundry Agent Service in the current documentation, focuses on the application and orchestration side. It provides a managed agent platform where we can select models, configure instructions, attach tools, call external services, and expose the final agent through an application workflow.

The important point is this:

That separation is useful because enterprise data work and AI application work are related, but they are not the same thing.

What Microsoft Fabric Data Agent Does

A Fabric Data Agent is designed to answer questions about data. In practice, the workflow usually looks like this:

Create a Fabric Data Agent
    |
    v
Connect data sources
    |
    v
Add business instructions
    |
    v
Test natural language questions
    |
    v
Improve the agent configuration
    |
    v
Publish the agent
    |
    v
Consume it from another application or agent

The data source can be something like a Lakehouse, Warehouse, Power BI semantic model, KQL database, mirrored database, or ontology, depending on the Fabric environment and the use case. Microsoft's Fabric Data Agent documentation describes the creation process as starting from a Fabric workspace and creating a new Data Agent item.

The value is not just that the agent can answer a question. The value is that the answer is grounded in enterprise data rather than the model's general knowledge.

For example, instead of asking a general chatbot:

What were our sales by country?

which the model cannot know, the user can ask a Fabric Data Agent that is connected to the organization's actual sales dataset.

The agent can then interpret the question, map it to the right data source, generate a query, retrieve the result, and return a structured response.

For business users, this is powerful because they no longer need to start every question from a dashboard, SQL query, or Excel export.

For technical teams, this is also useful because the data access pattern becomes more standardized. Instead of every department creating its own unofficial AI assistant, the organization can build data agents on top of managed Fabric assets.

Why Ontology Matters

One of the most important ideas in this architecture is ontology.

In Microsoft Fabric IQ, ontology defines business entities, relationships, properties, rules, and actions. Microsoft describes it as a shared business model that can be used across teams, agents, and workflows.

This matters because raw tables are not enough.

A table may contain columns like:

sales_amt
net_amt
gross_amt
region_cd
cust_id
txn_dt
fiscal_period

A business user does not think in those terms. They ask questions like:

Which market had the strongest revenue growth last quarter?

To answer correctly, the agent needs to understand the business meaning behind the data.

For example:

• Revenue may map to net sales, not gross sales.
• Market may map to region, not country.
• Last quarter may mean fiscal quarter, not calendar quarter.
• Growth may require comparing the current period with the same prior period.

Without a semantic layer, the model may still return an answer, but the answer may be based on the wrong field, wrong filter, or wrong aggregation.

That is dangerous because the output can look confident even when it is wrong.

Ontology helps reduce this risk by giving the agent a more structured understanding of the business domain.

A good ontology can define:

• Business entities: Customer, Product, Market, Sales Order, Revenue.
• Relationships: Customer belongs to Market, Sales Order contains Product, Revenue is generated from Sales Order.
• Properties: Customer segment, product category, fiscal period, sales amount.
• Rules: Revenue excludes cancelled orders, active customers must have at least one transaction in the selected period, fiscal quarter follows the company fiscal calendar.

This is where enterprise AI becomes much more than “chat with your data.”

The Role of AI Instructions

The next important layer is AI instructions.

If ontology tells the agent what the business concepts mean, instructions tell the agent how it should behave.

For example, a basic instruction could be:

You are a data analysis agent.
Answer only using the connected Fabric data sources.
If the answer cannot be found in the available data, say that the data is not available.
Do not guess.
When returning a number, explain the metric definition and aggregation logic.
If the question is ambiguous, ask a clarification question before answering.

These instructions are not just cosmetic. They shape how the agent handles uncertainty.

In a demo, it may be fine if the agent gives a rough answer. In production, that is not acceptable.

A production data agent should know when to say:

• I do not have enough information to answer that.
• This question could refer to either fiscal quarter or calendar quarter. Which one should I use?
• The connected data source does not include customer churn data.

This is a major difference between a consumer chatbot and an enterprise AI agent.

Publishing the Data Agent

After the Fabric Data Agent is configured and tested, the next step is publishing it so other systems can use it.

This is where the architecture becomes more interesting.

A Fabric Data Agent can be consumed through a Model Context Protocol endpoint. Microsoft documents Fabric Data Agent as an MCP server in preview, allowing external AI systems to consume the data agent through its MCP endpoint.

Conceptually, the flow becomes:

Fabric Data Agent
    |
    v
Published MCP Endpoint
    |
    v
External AI Agent or Application

This is important because it means the data agent does not need to remain trapped inside the Fabric UI.

Once published, it can become part of a larger AI system. For example, a company could use it in an internal analytics portal, Teams bot, Copilot Studio agent, Azure AI Foundry agent, Power Automate workflow, custom web application, or executive dashboard assistant.

This is the point where the agent starts moving from a demo into an actual product or workflow.

Connecting Fabric Data Agent with Azure AI Foundry

Azure AI Foundry becomes useful when we want to build a broader AI application around the data agent.

For example, the Foundry agent may need to:

• Receive a user question.
• Decide whether the question is about enterprise data.
• Call the Fabric Data Agent when needed.
• Use another tool for document search.
• Use another function for workflow automation.
• Summarize the result.
• Return a business-friendly answer.

Foundry Agent Service supports tools, and Microsoft's documentation describes agent tools as a way to connect agents to external data sources and capabilities.

A simplified workflow could look like this:

User asks:
"What were our top sales regions last quarter, and can you summarize the business implication?"

Azure AI Foundry Agent:
1. Understands the request.
2. Calls Fabric Data Agent for sales by region.
3. Receives structured results.
4. Interprets the trend.
5. Generates an executive-friendly summary.
6. Optionally suggests follow-up analysis.

The Fabric Data Agent handles the data retrieval. The Foundry Agent handles the orchestration and user experience.

That distinction is important.

The Foundry Agent should not randomly invent data. The Fabric Data Agent should not be expected to manage the entire application experience. Each layer has its role.

Example Implementation Pattern

A practical implementation pattern could look like this:

1. Prepare the data in Microsoft Fabric.
2. Create or reuse a semantic model.
3. Create a Fabric Data Agent.
4. Add instructions and test questions.
5. Improve the ontology or business definitions.
6. Publish the Data Agent.
7. Connect it to Azure AI Foundry.
8. Build a frontend or business workflow.
9. Add logging, monitoring, and governance.

For a small proof of concept, the first version does not need to be complicated.

A useful first version could support only five to ten business questions, such as:

• Show sales by country.
• Show revenue by product category.
• Compare this quarter with last quarter.
• Find the top five regions by revenue.
• Explain why total sales changed compared with last month.

The goal of the first version should not be to answer everything.

The goal should be to answer a small set of important questions reliably. That is a better enterprise AI strategy than building a broad agent that gives unpredictable answers.

Calling an Agent from Code

Once an agent is available through Foundry, developers can integrate it into an application using SDKs and endpoints. Microsoft provides Foundry SDKs and endpoint documentation for building AI applications on top of Foundry projects.

A simplified Python-style pattern looks like this:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

project_client = AIProjectClient(
    endpoint="YOUR_PROJECT_ENDPOINT",
    credential=DefaultAzureCredential()
)

agent = project_client.agents.get_agent("YOUR_AGENT_ID")

thread = project_client.agents.threads.create()

project_client.agents.messages.create(
    thread_id=thread.id,
    role="user",
    content="Show total sales by country"
)

run = project_client.agents.runs.create_and_process(
    thread_id=thread.id,
    agent_id=agent.id
)

messages = project_client.agents.messages.list(
    thread_id=thread.id
)

for message in messages:
    print(message)

In a real project, this would usually be wrapped inside an application backend.

Next.js frontend
    |
    v
API route or backend service
    |
    v
Azure AI Foundry Agent
    |
    v
Fabric Data Agent
    |
    v
Fabric data source

The end user does not need to know that Fabric or Foundry exists. They only see a clean interface where they can ask a business question and receive a governed answer.

Example Use Case: Sales Analytics Agent

Imagine a company wants to build a sales analytics assistant.

The user might ask:

Which countries had the highest sales last quarter?

The agent should not just return a number. A good enterprise response may include:

A better version may also provide:

• Compared with the previous quarter, Switzerland increased by 12%.
• Portugal decreased by 4%.
• The main driver of the Switzerland increase was higher sales in the enterprise customer segment.

This is where the solution becomes useful. The agent is not only retrieving data. It is turning data into an explanation.

But the explanation must still be grounded in the data. That balance is the core challenge of enterprise AI.

Example Use Case: Power BI Semantic Model Assistant

Another strong use case is building an assistant on top of an existing Power BI semantic model.

Many organizations already have Power BI datasets that contain curated measures, relationships, hierarchies, and business logic.

Instead of making users manually navigate reports, the agent can provide a conversational layer.

For example:

This kind of agent can be especially useful when the report has many pages, the user does not know which visual to open, the question is ad hoc, the user wants a quick explanation rather than a dashboard, or the business team wants self-service analytics.

However, this only works well if the semantic model is clean.

Poorly named measures, inconsistent relationships, and unclear definitions will reduce the quality of the agent's answers.

What Can Go Wrong

There are several risks that teams should take seriously.

Ambiguous Business Terms

If the user asks:

Show revenue by region.

The agent needs to know what revenue and region mean. If the company has multiple revenue measures, the agent may choose the wrong one unless the semantic model, ontology, or instructions are clear.

Weak Data Permissions

Enterprise data agents must respect user permissions.

A sales manager should not automatically see HR compensation data. A regional analyst should not automatically see global confidential metrics.

Security design should be part of the architecture from the beginning, not added at the end.

Overconfident Answers

LLMs can produce fluent answers even when the underlying data is missing or incomplete.

This is why the instruction should explicitly tell the agent not to guess.

If the data source does not contain the answer, say so clearly.

Lack of Testing

A data agent needs test cases.

Without test cases, teams may not notice when a prompt change, data model change, or permission change breaks the agent.

No Change Control

Once an agent is used in production, changes need to be tracked.

Teams should know who changed the instruction, who changed the data source, who published the latest version, what test cases were run, whether the output was reviewed, and whether rollback is possible.

This is where AgentOps and AI governance become important.

Why This Matters for Data Professionals

For data analysts, BI developers, and data scientists, this is a major shift.

Traditional analytics work often looks like this:

Collect requirements
    |
    v
Build data model
    |
    v
Create dashboard
    |
    v
Publish report
    |
    v
Answer follow-up questions manually

With AI agents, the workflow starts to change:

Build governed data model
    |
    v
Define business meaning
    |
    v
Create data agent
    |
    v
Test natural language questions
    |
    v
Publish agent endpoint
    |
    v
Integrate agent into workflow
    |
    v
Monitor answer quality

The skill set also changes. It is no longer enough to only know SQL, Power BI, or Python.

The next generation of data professionals will need to understand semantic modeling, ontology design, prompt and instruction design, agent testing, AI governance, API integration, security and permission design, and business workflow integration.

This does not replace traditional data skills. It builds on top of them.

In fact, strong data modeling becomes even more important because the agent depends on the quality of the underlying data structure.

My Practical Learning Path

If I were learning this stack from scratch, I would not start with a complex multi-agent system.

I would follow this path:

1. Learn the basics of Microsoft Fabric.
2. Understand Lakehouse, Warehouse, and Power BI semantic models.
3. Create a simple Fabric Data Agent.
4. Test it with a small set of business questions.
5. Improve the data model and instructions.
6. Learn how ontology works in Fabric IQ.
7. Publish the data agent.
8. Connect it to Azure AI Foundry.
9. Call the Foundry agent from a small web app.
10. Add logging, evaluation, and governance.

The key is to start small.

A reliable agent that answers five business questions correctly is more valuable than a broad agent that answers fifty questions unreliably.

A Simple Mental Model

Here is the simplest way I think about the stack:

Or even shorter:

Together, they create a path from governed enterprise data to production AI applications.

Final Thoughts

The most interesting part of Azure AI Foundry and Microsoft Fabric Data Agent is not that they make it possible to build another chatbot.

The interesting part is that they move AI closer to real enterprise workflows.

A useful enterprise agent needs to do more than generate text. It needs to understand business definitions, respect data permissions, use governed data sources, ask clarification questions, avoid guessing, return traceable answers, fit into existing applications, and support testing and change management.

That is why I think this architecture is worth paying attention to.

For many organizations, the future of analytics may not be only dashboards. Dashboards will still matter, but they will be joined by conversational data agents that help users ask questions, explore metrics, and understand business performance in a more natural way.

The real opportunity is not simply “chat with data.”

That is the direction I believe data teams should start preparing for now.

References

Product names, preview features, and integration details can change quickly, so I used current Microsoft Learn pages as source links for the platform details in this article.

• Fabric data agent overview
• Create a Fabric data agent
• Data agent as a Model Context Protocol server
• Consume Fabric data agent from Microsoft Foundry Services
• What is Microsoft Foundry?
• What is Microsoft Foundry Agent Service?
• Agent tools overview for Microsoft Foundry Agent Service
• What is Ontology in Microsoft Fabric IQ?