As I'm sure you already know, AI responses aren't always correct (your favorite LLM probably has a disclaimer saying the same thing). So as adoption of AI coding agents like Claude Code—or really any application that calls tools—takes off, it's natural to have questions about the answers users get back. Did the agent call the right tool? What context went to the LLM before it gave its response?
Traditional application monitoring can tell you that an API responded successfully within 5 seconds. That's useful! But when that observability philosophy is applied to AI, it treats the entire workflow (whether it's agentic, RAG, or beyond) as a single black box.
That's why AI observability matters: it makes it possible to connect the response a user sees to the model calls, retrieved context, tool uses, and all the other steps that produced it. In this blog post, I’ll walk through a typical agentic application and show you how that can work, using the open source project MLflow behind the scenes.
Where you can go wrong without AI observability
Say we have a mortgage lending application, where our team can ask an AI assistant questions about the business—something like "show me the current pipeline status." The assistant uses tools that retrieve internal data.
Figure 1 illustrates the dashboard where team members can see data and interact with the assistant. At first glance, the assistant’s answer looks solid. But notice that its 44% pull-through rate and 60 average days to close response doesn't match the dashboard, which gives 43.2% and 52.5 days for the same metrics.
So did the model hallucinate? Well… we don't know! The dashboard and the assistant could be using different time windows, different queries, or different definitions of "pull-through." From this view, there's no way to tell. That's the observability gap, and the way out of it is being able to see the path that produced the answer.
Finding the request in MLflow
All we've seen so far is a final answer, but there was an entire decision-making process that happened in the few seconds between the question and the response. A typical agentic query like this one generates more than 30 metrics (token counts, LLM latency, database query times) and more than 15 log events (agent decisions, tool parameters, and so on) into a single distributed trace. MLflow can provide visibility into those traces, as shown in Figure 2.
This gives us a lot more than an HTTP access log. We can associate a request with a conversation session, inspect its complete response, and open the trace directly instead of manually correlating a handful of services, as shown in Figure 3.
In this specific trace, we can see:
- The first model call took 2.48 seconds and decided what action to take.
- A tool call (
ceo_pipeline_summary
) grabbed data from our internal application and returned it to the LLM in 75 milliseconds. - A second model call took 2.40 seconds and composed the final answer.
So the tool wasn't the bottleneck here. Almost all of the request time came from the 2 model calls. That's immediately useful to 2 different teams: your SRE/platform folks get real latency attribution, and your AI engineering team gets a view into model behavior and decision quality.
Tracing what happened to the AI agent
Now let's look inside those spans, specifically the ceo_pipeline_summary
tool call we mentioned earlier. Figure 4 illustrates what this looks like.
There it is! MLflow shows that our agent received an input of 44% and 60 days, the exact numbers it reported. So we have concrete evidence the model didn't invent those figures. It accurately summarized what it received from the tool.
This means that the difference between our assistant and the dashboard is farther upstream. Maybe the data is calculated differently, maybe one result was cached, maybe the tool is querying a different date range. We still don't know the answer! But we've gone from "the AI got it wrong" to "the tool and the dashboard disagree," and that's a bug someone can actually pick up and fix. That's the practical value of tracing an agent.
What MLflow adds to your observability stack
Simply put, we need metrics, logs, and traces more than ever. MLflow connects a single user request to the model calls, tools, prompts, and context that produced an answer.
MLflow is open source and compatible with OpenTelemetry, and also with automatic tracing integrations for popular model and agent frameworks, including LangChain, Claude Code, and many other AI tools.
With Red Hat OpenShift AI, you get logical isolation for your traces and experiments, so alongside serving models and running automations, you can debug requests and watch how your AI applications and agents are actually performing, in one place, across any hybrid cloud provider. That's pretty neat, I'd say! You can try out the demo application we’ve been discussing (see Figure 5 for the architecture) as a Red Hat AI Quickstart. It demonstrates a complete (and sovereign) agentic AI use case with MLflow.
Wrapping up
We just walked through debugging a single request, but the real value shows up over time. Once you're capturing traces, you can use them to evaluate future versions of the application, watch for changes on your model provider's end (did it just get nerfed?), and build pipelines that catch regressions in tool calls before your users do.
As we depend more and more on AI applications and agents, I really think AI observability stops being just nice to have and becomes a requirement for anything you put in production. It makes it possible to either confirm that you’re getting the right answer back, or to figure out exactly where you went wrong.