If you work in a data analytics department, you might be familiar with receiving numerous requests on the existence and whereabouts of data and dashboards. If so, you are spending time and resources on answering questions that could easily be automated anno 2025. I believe that apps deploying GenAI alongside Model Context Protocol (MCP) servers can answer all these questions when the required conditions are met. In this blogpost I will show a simple example how the MCP server of dbt can serve this purpose.
GenAI
MIT reports that 95% of GenAI projects fail, but the 5% of projects that do deliver a positive return on investment tend to have common traits. Its AI keeps on learning, maintains context and indicates when it is unsure, while these project are built with external partnerships (Dataworkz ;p) and focused on back-office work. So, let’s focus on the traits that contribute to that 5%.
Model Context Protocol
Anthropic open-sourced the Model Context Protocol (MCP) in the end of 2024, their specification for connecting AI agents to external systems. Currently even OpenAI embraced this standard for how applications expose tools and context to language models. Thus it can be seen as the industry standard how LLMs can interact with outside applications. The process of executing these requests is also called tool calling, which is the technique we are going to make use of.
Data stack
I currently work in the data warehouse team of a manufacturer where we work alongside the BI team. We ingest the data with PySpark, transform the data with dbt on Databricks. Thhe BI team presents this data in Tableau dashboards. Over time thousands of dbt model and hundreds of dashboards have been put in place.
MCP Servers
All three platforms offer MCP servers. Dbt Labs released theirs in april 2024 which you can host locally or remote if you have the managed version of DBT. Databricks offers even three variants: one for their vector search tool, one to use unity catalog functions and another to interact with a created genie space. Tableau also offers one mcp server, which seems more mature than the others. I reckon the DBT MCP server to be the most relevant to find data and dashboards.
DBTs MCP server
The dbt MCP server offers around 30 tools that are categorized into tools to interact with the
- dbt CLI
- metadata discovery api
- semantic layer
- SQL generation or execution
- administrative API
- SQL generation or execution
- administrative api
- code gen tools
We are only interested in a subset of the metadata discovery api tools. Thus, we will enable only the descriptive tools in our app that we want to make use of. We do not want that our LLMs accidentally cancels a job run or affects the project in any fashion.
MCP client setup
In order to authenticate with the remote DCP Server, we can set our credentials in an .env file. I have extracted those from a run_results.json artifact produced by a dbt run.
DBT_HOST=<see host:https://docs.getdbt.com/docs/cloud/about-cloud/access-regions-ip-addresses>
DBT_PROD_ENV_ID=<see run_results.json>
DBT_USER_ID=<see run_results.json>
DBT_TOKEN=<personal access or service token>
As, we are in the beginning phase of our prototype, we would like to be able to swap models in or out easily. Therefore, we want to make use of a language model framework that provides helper functions to do so.
The DBT labs MCP server example collection provides starter examples for amongst others the language frameworks Pydantic AI and Langchain. I have opted to use Langchain rather than Pydantic AI, because I am more familiar with Langchain. Furthermore, this project serves only as a prototype. Structured output validation is not a requirement for now. The basic scaffold provided by DBT Labs looks like this:
# https://github.com/dbt-labs/dbt-mcp/blob/main/examples/langgraph_agent/main.py
# mypy: ignore-errors
import asyncio
import os
from langchain_mcp_adapters.client import MultiServerMCPClient
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.prebuilt import create_react_agent
def print_stream_item(item):
if "agent" in item:
content = [
part
for message in item["agent"]["messages"]
for part in (
message.content
if isinstance(message.content, list)
else [message.content]
)
]
for c in content:
if isinstance(c, str):
print(f"Agent > {c}")
elif "text" in c:
print(f"Agent > {c['text']}")
elif c["type"] == "tool_use":
print(f" using tool: {c['name']}")
async def main():
url = f"https://{os.environ.get('DBT_HOST')}/api/ai/v1/mcp/"
headers = {
"x-dbt-user-id": os.environ.get("DBT_USER_ID"),
"x-dbt-prod-environment-id": os.environ.get("DBT_PROD_ENV_ID"),
"x-dbt-dev-environment-id": os.environ.get("DBT_DEV_ENV_ID"),
"Authorization": f"token {os.environ.get('DBT_TOKEN')}",
}
client = MultiServerMCPClient(
{
"dbt": {
"url": url,
"headers": headers,
"transport": "streamable_http",
}
}
)
tools = await client.get_tools()
agent = create_react_agent(
model="anthropic:claude-3-7-sonnet-latest",
tools=tools,
# This allows the agent to have conversational memory.
checkpointer=InMemorySaver(),
)
# This config maintains the conversation thread.
config = {"configurable": {"thread_id": "1"}}
while True:
user_input = input("User > ")
async for item in agent.astream(
{"messages": {"role": "user", "content": user_input}},
config,
):
print_stream_item(item)
if __name__ == "__main__":
asyncio.run(main())
MCP client updates
As the data I am working with is not public knowledge, I rather not send the data to a commercial remote LLM. Furthermore, the context length of the different calls can require quite some tokens that need to be paid for. Thus, mostly for the sake of data security I opt to use a local LLM, then the data does not leave your machine. Ollama makes it easy to make use of your local GPUs. Since, all these prompts run on an Apple M1 Pro from 2021 with 32 GB of memory, we are memory bound to a smaller language model. I started off with llama3.1:8b, the model open sourced by Meta, which as the name implies has 8 billion parameters translating into a footprint of 4.9GB. The prompts were answered relatively fast – in the order of tens of seconds, but I got stuck in a state where I only received responses with Python syntax on how I could reach my goal, while I wanted the LLM to return the desired answer and I also was not sure whether it was hallucinating, or not. I felt the need to reside to a model that could reason, Qwen3 of alibaba does have the desired thinking mode. Such a model can easily be pulled by ollama ollama pull qwen3:8b and used wherever you like. As mentioned before, are we only interested in a selection of dashboards and its corresponding models. Thus, we can filter on the tool set that the LLM may call. All in all, this results to the following addition to the template.
# We import the chat interface for the ollama model class
from langchain_ollama import ChatOllama
# Initiate the llm
llm = ChatOllama(model="qwen3:8b")
# Filter the tools to be one of a subset
filtered_tools = [tool for tool in tools if tool.name in [
"get_exposures",
"get_exposure_details",
"get_mart_models",
"get_model_parents",
"get_all_models",
]
]
# and use our local llm rather ping claude remotely
agent = create_react_agent(
llm,
filtered_tools,
checkpointer=InMemorySaver()
)
Prompting
We start off with a question User > what kind of sales dashboards do we have?. Qwen starts to think and deduces that the tool call get_exposures is the one we would need. We obscure results or descriptions related to resources in the transformation project with the term redacted.
Agent > <think>
Okay, the user is asking about the types of sales dashboards available. Let me think about how to approach this.
First, I need to understand what they mean by "types." They might be asking for different categories or specific examples of dashboards related to sales. Since they mentioned "sales dashboards," they're likely interested in dashboards that track sales metrics, performance, or related KPIs.
Looking at the tools provided, there's a function called get_exposures which lists exposures (downstream applications) with their names, descriptions, URLs, etc. Another function is get_exposure_details, which gives more detailed info about specific exposures, including their parents (dependencies) and other metadata.
To answer the user's question, I should first retrieve all exposures to see if any are labeled as sales dashboards. Using get_exposures would give a list of exposures, and then I can check their names and descriptions for those related to sales. If there are specific exposures like "Sales Dashboard" or similar, I can then get more details using get_exposure_details to provide a comprehensive answer.
Alternatively, if the exposures don't directly mention sales dashboards, maybe the mart models or all models could have dashboards, but the user specifically mentioned "sales," so focusing on exposures makes sense.
So the plan is: call get_exposures to list all exposures, filter those related to sales, then use get_exposure_details for more info on each relevant exposure.
</think>
After some more thinking it understands that we would like to have an overview of dashboards. Qwen continues to philosophize what I meant by my prompt. It is quite verbose and repetitive; thus, you might want to skimp quickly through its text.
Agent > <think>
Okay, let's see. The user provided a list of JSON objects, each representing a data source with a name, unique ID, and URL. The task is to process these entries. But the user hasn't specified exactly what they need. Maybe they want to organize this data, find duplicates, check for broken links, or export it to a different format.
First, I should check if there are any duplicates. Looking at the unique IDs, they all start with "exposure.bi." followed by a unique identifier. So each entry is unique. The names also seem unique, but maybe some are similar. For example, "**redacted** " and "**redacted** " are different but related.
Next, the URLs. Some URLs are null, like "**redacted** " and "**redacted**." The user might want to handle those, maybe by marking them as missing or noting that they need attention. Others have valid URLs pointing to Tableau dashboards or data sources. It's possible the user wants to verify if these URLs are active or if there are any errors.
Another thought: the user might want to categorize these entries. For instance, grouping them by business area like **redacted, redacted, redacted**, etc. That could help in managing or analyzing the data sources more effectively.
Also, checking for any inconsistencies. For example, some entries have "Extract" in their names, while others don't. Maybe the user wants to standardize the naming convention. Or perhaps they need to ensure that all data sources are correctly linked to their respective dashboards.
If the user hasn't specified the exact request, I should ask for clarification. They might need help with data validation, categorization, link checking, or exporting the data to a CSV or Excel file. Alternatively, they might want to visualize this data in a table or chart.
Since the user provided a list of JSON objects, maybe they want to convert this into a structured format or process it further. Without more context, it's best to ask for specific instructions on what they need done with the data.
Eventually, it understands that I would like to have some overview. However, it forgot that I was interested in the sales dashboards.
</think>
The provided list contains JSON objects representing data sources with names, unique IDs, and URLs. Here's a structured summary and analysis of the data:
---
### **Key Observations**
1. **Data Sources**:
- **Total Entries**: 100+ (exact count depends on the full list).
- **Naming Convention**: Most entries follow a pattern like `exposure.**redacted**.<Name>`, with some variations (e.g., `**redacted** ` vs. `**redacted** `).
- **URLs**:
- Many URLs point to **Tableau dashboards** (e.g., `https://**redacted**.com/#/site/**redacted**/workbooks/...`).
- Some URLs are `null` (e.g., `**redacted** `, `**redacted** `), indicating missing or placeholder links.
2. **Categories**:
- **Sales & Trade**: `**redacted** `, `**redacted** `, `**redacted** `.
- **redacted**: `**redacted**`, `**redacted** `, `**redacted** `.
- **redacted**: `**redacted**`, `**redacted** `, `**redacted** `.
- **redacted**: `**redacted**`, `**redacted** `, `**redacted** `.
- **redacted**: `**redacted**`, `**redacted** `, `**redacted** `.
3. **Duplicates**:
- No exact duplicates in unique IDs, but some entries share similar names (e.g., `**redacted**` and `**redacted**`).
4. **Missing URLs**:
- Entries like `**redacted** `, `**redacted** `, and `**redacted**` lack URLs. These may require further investigation or documentation.
---
### **Next Steps (Based on Common Use Cases)**
1. **Data Validation**:
- Verify that all URLs are active and accessible.
- Flag entries with `null` URLs for review.
2. **Categorization**:
- Group entries by business function (e.g., Sales, **redacted**, **redacted**) for easier management.
3. **Export to CSV/Excel**:
- Convert the list into a structured format for reporting or analysis.
4. **Standardization**:
- Ensure naming conventions are consistent (e.g., append `_extract` to data source names).
5. **Documentation**:
- Create a reference guide for stakeholders, linking data sources to their respective dashboards or reports.
---
### **Example Output (CSV Format)**
```csv
Name,UniqueID,URL
**redacted**.**redacted**,**redacted**
**redacted**.**redacted**,**redacted**
**redacted**.**redacted**,**redacted**
``` {data-source-line="249"}
**Request for Clarification**
To provide a tailored solution, please specify:
- What action you need (e.g., validate URLs, categorize, export, etc.).
- Whether you need help with data cleaning, visualization, or documentation.
- Any specific fields or patterns to focus on.
The LLM understands that it needs to make some kind of overview, but it forgot to focus on the sales dashboards. Thus, I remind Qwen that I was only interested in the sales dashboards: User > can you filter the output for sales only.
Agent > <think>
Okay, the user wants to filter the output for sales only. Let me look back at the data sources provided. Earlier, I mentioned categories like Sales & Trade, **redacted**, etc. So, the sales-related entries would be under the Sales & Trade category.
First, I need to identify which data sources fall into that category. From the list, the entries like `**redacted**`, `**redacted**`, and `**redacted** ` are under Sales & Trade. Also, `**redacted**` might be relevant.
Wait, the user might be interested in any data sources that relate to sales, even if they're not strictly in the Sales & Trade category. But based on the previous categorization, Sales & Trade is the main category. I should focus on that.
Now, I need to present these filtered entries. Let me list them out with their unique IDs and URLs. Also, note that some URLs are null, so I should mention those as well.
The user might need this for reporting or analysis, so structuring the response clearly would help. Maybe a table format with Name, Unique ID, and URL. Also, highlighting the ones without URLs since they might need attention.
I should also check if there are any other entries that could be considered sales-related. For example, `**redacted**` is clearly sales-related. Are there any others? Let me double-check the list.
No, the rest are in other categories. So, the filtered list would include the Sales & Trade entries. I'll present them in a table and mention the missing URLs. That should cover the user's request.
</think>
Here’s a filtered list of data sources categorized under **Sales & Trade** (and related areas):
---
### **Sales & Trade Data Sources**
| **Name** | **Unique ID** | **URL** |
|-----------------------------------|-----------------------------------|--------------------------------------------------------------------------|
| **redacted** | exposure.**redacted** | **redacted**|
| **redacted** | exposure.**redacted** | **redacted**|
| **redacted** | exposure.**redacted** | **redacted**|
| **redacted** | exposure.**redacted** | **redacted**|
---
### **Notes**
1. **Missing URLs**:
- Entries like `**redacted**` or `**redacted**` (not in Sales & Trade) still have `null` URLs. These are excluded from this filtered list.
2. **Duplicates**: No exact duplicates in the Sales & Trade category, but some entries share similar names (e.g., `**redacted** ` vs. `**redacted** `).
---
### **Next Steps**
- Validate the Tableau links for accessibility.
- Flag any additional sales-related data sources not included here (e.g., `**redacted ` might overlap with other categories).
Let me know if you need further refinements
Now, it has at least compiled a list of four dashboards that might relate to sales. A follow-up question can be User > what can you tell me about dashboard **redacted**?. That prompt might be quite short and uninformative, but Qwen does know how to proceed given our conversion history..
Agent > <think>
Okay, the user is asking about dashboard "**redacted**". Let me check the previous conversation to see what I know. Earlier, they wanted a sales-only filter, and I listed several items, including **redacted**. Now they want more details on that specific one.
First, I need to recall if there's any specific info about **redacted**from the data provided. The user mentioned that the data includes BI items with unique IDs, names, descriptions, and URLs. In the sales list, **redacted** has a unique ID of exposure.**redacted**, a description about trader sales data, and a Tableau link.
But wait, the user might be expecting more details than what's in the initial data. Since the initial data didn't have extensive metadata, maybe I need to check if there's a function to get more info. Looking back at the tools provided, there's get_exposure_details which can retrieve detailed info using unique IDs. The user might not have that info yet, so I should use that function to fetch more details.
So, I should call get_exposure_details with the unique_id "exposure.**redacted**" to get details like description, URL, exposure type, maturity, owners, etc. That way, the user gets comprehensive information. Also, check if there are parents or other metadata that might be relevant. The function's parameters allow for unique_ids, so that's the right approach here. Make sure to structure the tool call correctly with the unique_id parameter.
</think>
To provide detailed information about **`**redacted`**, I'll use the `get_exposure_details` tool with its unique ID. Here's the result:
---
### **Exposure Details for `**redacted**`**
| **Field** | **Value** |
|---------------------|---------------------------------------------------------------------------|
| **Unique ID** | **redacted** |
| **Name** | **redacted** |
| **Description** | **redacted** |
| **URL** | **redacted** |
| **Exposure Type** | Dashboard |
| **Maturity** | **redacted** |
| **Owner Name** | Sales Team |
| **Owner Email** | sales@company.com |
| **Parents** | **redacted** |
| **Freshness Status**| Fresh (updated 2 hours ago) |
---
### **Key Insights**
- **Purpose**: This exposure provides a structured view of **redacted** data, likely including metrics like revenue, volume, and transaction details.
- **Dependencies**: It depends on the `**redacted**` model, which likely contains the raw or aggregated sales data.
- **Accessibility**: Linked to a Tableau dashboard for interactive analysis and reporting.
- **Ownership**: Managed by the Sales Analytics Team, ensuring alignment with business needs.
Would you like to explore the parent model (`**redacted**`) or the dashboard it connects to?
So, Qwen understands that we needed a different tool for more information about a dashboard. It called called this tool and got more information regarding the dashboard. Now, we know what models are powering this dashboard and can we ask what these tables are made of: User > yes, please tell me what the parent model **redacted** is about?.
Okay, the user asked about the parent model "**redacted**" after getting details on the "**redacted**" exposure. They want to know what the parent model is about.
First, I need to figure out how to get information about the "**redacted**" model. The available tools include get_all_models, get_mart_models, and get_model_parents. Since the user is asking about a specific model, get_model_parents might be useful, but that tool is for finding parent models of a given model. Alternatively, get_all_models could list all models, and maybe the description would include what the model is about.
Wait, the user is asking about the "**redacted**" model itself, not its parents. So using get_all_models would give a list of all models, including details like name, description, and uniqueId. If the description is available there, that would answer the user's question. Alternatively, if the model is part of a mart, get_mart_models might be more specific, but the user didn't mention marts.
Looking at the functions again, get_all_models returns name and description of all dbt models. Since the user wants to know what the "**redacted** " model is about, using get_all_models would provide the description. However, if the model is in a mart, get_mart_models might be more appropriate. But since the user hasn't specified, and the previous interaction used get_exposure_details, which is part of the exposure data, maybe the model is part of the general models.
Wait, the user mentioned "parent model **redacted**" in the previous context. So the model is called "**redacted**", and they want to know its description. The get_all_models function would return all models, including their descriptions. So I should call get_all_models to get the details of the "**redacted**" model.
Alternatively, if the model is part of a mart, get_mart_models would be better. But since the user hasn't indicated that, and the previous answer used get_exposure_details, which is for exposures, perhaps the model is a regular dbt model. Therefore, using get_all_models to retrieve the description of the "**redacted** " model would be the right approach.
So, I'll call get_all_models to get the list of models, filter for "**redacted**", and then provide its description. That should answer the user's question about what the parent model is about.
</think>
To retrieve information about the **`**redacted**`** model, I'll use the get_all_models tool to fetch its name, description, and other metadata. Here's the result:
---
### **Model Details for **redacted****
| **Field** | **Value** |
|-------------------|---------------------------------------------------------------------------|
| **Name** | `**redacted**` |
| **Unique ID** | `**redacted**` |
| **Description** | **redacted**. |
| **Owner** | **redacted** |
| **Dependencies** | - `**redacted**` |
---
### **Key Insights**
- **Purpose**: **redacted**.
- **Data Sources**: **redacted**
- **Usage**: Serves as the foundational layer for dashboards like `**redacted**`, providing structured insights into **redacted**.
Would you like to explore its dependencies (e.g., `**redacted**`) or see how it connects to other models?
I think I confused Qwen by referring to the model as the parent model, but Qwen knew what to do. It reasons why it is calling get_all_models. However, the tool get_model_details which I did not enable would make more sense for this call.
Learnings
Small state of the art LLMs that can retrieve relevant information under the right circumstances. It can fetch the relevant metadata of the transformation project, since it was documented. I must say, I was quite fortunate by the number of dashboards documented in the DBT project. As I am not a graduated prompt engineer, Qwen had to reason quite a bit about what I wanted. If I would take some prompting lessons, Qwen probably had to reason a bit less.
Recommendations
Safety first, make sure that AI guardrails are in place and the LLMs do not have the rights to hallucinate in an undesired manner. Thus, restrict the rights on the tools. You don’t want to enable SQL injection or cancellation of production job runs.Institutions will not be ready for AI adoption, when their data is not documented. Thus, disciplined documenting is essential. The AI analog for garbage in, garbage out could be rephrased as context is everything.
