The infrastructure dashboard is green.

Availability sits above target. Latency is within its normal band. HTTP error rates look unremarkable. The database answers quickly. The model provider is responding without incident. Nothing in the application logs suggests an exception worth escalating. By every convention that engineering organizations have spent two decades refining, the product is healthy.

And yet something is wrong.

Users are abandoning conversations with the AI assistant partway through. Support teams notice that responses feel less useful than they did a few weeks ago, though nobody can point to a specific defect. One customer mentions, almost in passing, that the system keeps asking questions it should already know the answers to. A workflow that used to resolve in a handful of steps now seems to take longer, though no one changed it. Finance flags that AI infrastructure spending is climbing faster than usage would suggest. No single service is broken. No alert has fired. That is exactly the problem.

This is not a story about an outage. It is a story about the growing distance between what a dashboard says and what a product is actually doing — a gap that barely mattered for most of the history of software, and that now matters enormously.

The Green Dashboard Fallacy

Call it the Green Dashboard Fallacy: the assumption that healthy infrastructure implies healthy product behavior.

For traditional, deterministic software, this assumption was never entirely true — a web application could be fully available while a business-logic bug silently miscalculated a discount, or a batch job could complete successfully while writing incorrect records. But the gap between "it ran" and "it worked" was usually narrow, and usually caused by a bug that could be found, reproduced, and fixed once. Execution and correctness were tightly coupled enough that engineering teams could reasonably treat uptime, latency, and error rate as decent proxies for whether the software was doing its job.

AI-powered systems break that coupling in a more fundamental way. A model request can return a perfectly successful response and still produce a poor answer. A vector database query can execute flawlessly and still retrieve context that has nothing to do with the question being asked. A tool call can complete without error while being the wrong tool for the situation. A multi-agent workflow can finish with zero exceptions after taking twelve steps that should have taken three. A routing layer can send a request exactly where it was configured to send it, and that configuration can quietly be burning money on a model far more capable than the task requires.

None of these are infrastructure failures. Every component involved did what it was told to do. The failure lives somewhere else — in the gap between successful execution and acceptable behavior. Conventional health checks were never built to see that gap, because for most of software's history, that gap barely existed.

From "Did It Run?" to "What Did It Do?"

Traditional monitoring is organized around execution status: request received, function executed, database queried, response returned, status code logged. This is a reasonable design when the software's behavior is fully specified by its code. If the code is correct and it ran, the outcome is very likely correct too.

AI systems weaken that inference. Consider a customer-support agent handling a single conversation turn. Infrastructure telemetry might show one inbound request, three model calls, two database queries, a successful response, and a total latency of 2.4 seconds. That telemetry is accurate, complete, and almost useless for answering the question that actually matters: did the agent understand what the customer needed, retrieve the right information, apply the correct policy, choose an appropriate tool, and answer correctly — or escalate when it should have?

Execution needs interpretation now. The trace of what happened during a request has become at least as important as whether the request technically succeeded, because "succeeded" no longer implies "was right."

The Observability Gap

This distance has a shape, and it is worth naming: the Observability Gap is the distance between what conventional telemetry can tell an organization and what that organization actually needs to know about how its AI is behaving.

The gap is not fixed. It grows with architectural complexity. A single LLM completion embedded in a simple feature has a fairly small observability gap — there is one call, one input, one output, and a human can often eyeball whether it looks reasonable. A multi-agent workflow that pulls from a CRM, a billing system, an internal knowledge base, and a handful of external APIs has a dramatically larger one, because there are now many more places where behavior can diverge from intent, and far fewer opportunities for a human to notice.

This matters for how organizations think about their own systems. The amount of AI observability a product needs is not a fixed cost of "having AI." It is a function of how much decision-making surface the architecture actually contains. A product that wraps a single well-scoped prompt around a narrow task has different needs than one that lets an agent choose its own tools, retrieve its own context, and decide when it is finished. Architecture drives observability requirements as much as risk does — often more, because architecture is chosen early and its observability implications are rarely considered at that stage.

The Control Room Was Built for Deterministic Machines

None of this is an argument that traditional monitoring has failed or become obsolete. It is worth being precise about what it got right.

Modern engineering organizations became genuinely excellent at monitoring servers, distributed systems, microservices, containers, databases, APIs, and cloud infrastructure. The vocabulary that grew out of that work — logs, metrics, traces, service-level indicators, service-level objectives, error budgets — produced real, measurable improvements in production reliability over the past two decades. Teams learned to detect degradation before customers noticed it, to correlate failures across services, and to hold themselves accountable to numeric targets rather than vague impressions of "things feel okay."

AI-powered applications still depend on every layer of that stack. They still run on networks, databases, APIs, compute, queues, storage, and cloud infrastructure. If the vector database goes down, the AI product goes down. If the network partitions, nothing an agent decides matters, because nothing can execute. Traditional observability remains the substrate — the foundation everything else stands on, and a foundation that has to hold or nothing above it means anything.

The mistake is not building this substrate. The mistake is treating it as the entire building. AI introduces behavioral layers on top of the substrate that the substrate was never designed to see, and no amount of additional server metrics will make those layers visible.

Availability Is Now Only the First Question

A useful way to see the shift is to notice how the questions an AI product needs to answer keep multiplying, each one sitting on top of the last.

Is the service available? That is where monitoring has always started, and it still has to be answered first — nothing else matters if the answer is no.

Is the model responding? A layer up from basic availability: the provider might be reachable while returning errors, throttling requests, or degrading in ways that don't trip a conventional uptime check.

Is the correct model responding? In systems with routing logic, the model that answers a given request is itself a variable, and getting that variable wrong is invisible to any check that only confirms "a model answered."

What context did it receive? The same prompt template can produce very different behavior depending on what was assembled into the context window before the model ever saw it.

What did it retrieve? For systems built on external knowledge, this question sits underneath almost everything else that goes wrong.

Which tool did it choose? Once an agent can act, its choice of action becomes as consequential as its choice of words.

Why did the workflow take eight steps? Path length and shape are themselves signals, not just a footnote to latency.

Was the result good? This is where telemetry ends and evaluation has to begin.

Did the user achieve what they wanted? The only question that ultimately matters commercially, and the hardest one to answer directly.

How much did the interaction cost? A question that used to live entirely in finance and now has to live in engineering too.

This progression is the spine of everything that follows. It moves from machines toward outcomes, and each step requires a different kind of evidence than the one before it.

The New Production Unit Is the Trace

If a single request-response pair is no longer a sufficient unit of observation, something has to take its place. That something is the trace — a record of everything that happened between a user's request and the system's final answer.

A useful AI trace captures the user's request, the application's orchestration logic, every model call made along the way, retrieval steps, tool invocations, agent handoffs, retries, fallbacks, the final output, timing at each stage, and token usage. In an agentic system, a trace effectively becomes a map of the decision process the application walked through to arrive at its answer, not just a record that an answer arrived.

This matters because the final answer, taken alone, tends to hide the cause of a problem rather than reveal it. If a response is wrong, the underlying cause could be flawed model reasoning, irrelevant retrieved context, stale documentation, a bad tool result, a routing mistake, context truncation, a broken prompt change, a failed fallback that silently continued, or bad external data feeding the whole pipeline. Without visibility into execution, every one of these collapses into the same unhelpful diagnosis: "the AI gave a bad answer." That sentence describes a symptom. It does not describe anything an engineer can act on.

The Same Bad Answer Can Have Ten Different Causes

It's worth sitting with one concrete example, because the point is easy to state abstractly and easy to miss in practice.

Suppose an assistant tells a customer the wrong cancellation policy. The customer-facing symptom is identical no matter what produced it, but the underlying cause could be any of several very different things, each demanding a different fix.

The model may have invented the answer outright, unsupported by anything it was given. Or retrieval found an outdated version of the policy document, in which case the fix is a data problem, not a model problem. Or retrieval found the correct document but ranked it too low to make it into the context window, which is a ranking problem rather than a coverage problem. Or the prompt excluded an instruction that would have steered the model correctly, which is a prompt-engineering regression. Or a routing layer sent the request to a model unsuited for nuanced policy language. Or an internal policy service itself returned stale data, meaning the AI layer did nothing wrong at all. Or the conversation's memory carried forward an earlier, outdated statement about the policy. Or the context window was too full and the relevant passage got truncated before the model ever saw it. Or a tool call failed silently and the agent continued as though it had succeeded. Or the customer's original question was ambiguous and the system guessed instead of asking for clarification.

Ten plausible causes, one identical symptom, and ten different operational responses. This is why AI observability has to be built to move an organization from symptom to behavioral cause — not because it's intellectually satisfying, but because "the AI gave a bad answer" is not a sentence anyone can act on, and repeating it after every failure is how organizations end up making the same mistake indefinitely while believing they are debugging it.

Tracing Is Not the Same as Logging Everything

There is an obvious temptation once the value of tracing becomes clear: capture everything, always, in full detail. That temptation should be resisted.

AI traces can contain exactly the kind of content organizations spend enormous effort protecting elsewhere — customer messages, internal documents, personally identifiable information, confidential business data, and in regulated industries, information subject to specific legal handling requirements. A trace store that captures every prompt and every response in full, retained indefinitely, with broad internal access, is not simply a debugging asset. It is also a growing liability surface that looks nothing like a server metrics dashboard and everything like a data warehouse full of customer conversations.

The trade-off is real and it does not resolve itself: more visibility tends to improve debugging, and more retained content tends to increase security and compliance exposure. Organizations navigate this with a handful of complementary techniques — redacting sensitive fields before storage, sampling rather than capturing every trace in full, preferring metadata-only telemetry where full content isn't needed, capturing content selectively based on risk, applying access controls so that raw traces aren't broadly readable, and setting retention policies that don't default to "forever." None of these is a universal answer, because what counts as acceptable retention varies by industry, by jurisdiction, and by the sensitivity of what a given product actually handles. The right posture is one an organization arrives at deliberately, with its legal and security functions involved, rather than one it backs into by turning on maximum logging and hoping nothing goes wrong.

Latency Is No Longer One Number

Traditional performance dashboards tend to report a single latency figure per request, and for a long time that was a reasonable simplification. AI applications complicate it in ways that make a single number actively misleading.

A user interacting with an AI system may experience several distinct forms of delay: the time before anything appears at all, the time the model spends generating tokens, retrieval latency, tool latency, the coordination overhead of a multi-agent workflow, delay introduced by retries, queueing time, and the total duration of a streamed response. Two requests with identical end-to-end latency numbers can feel completely different to a user — one that begins responding almost immediately and streams steadily feels responsive even if it takes a while to finish; one that sits blank and then dumps a full answer at once feels slow even if its total time is shorter. Time to first useful response deserves to be tracked as its own signal, separate from total completion time.

Agentic systems add a further wrinkle. A workflow that takes a long time might be doing so because it is reasoning carefully through a genuinely difficult task, or because it is stuck cycling through steps that aren't making progress. Both look identical on a raw latency chart. Only execution-level visibility can tell them apart, and the operational response to each is completely different — one is working as intended and simply slow, the other is a bug wearing the costume of thoroughness.

Fast and Wrong Is Not a Performance Win

Infrastructure teams have a long-standing, well-earned instinct to celebrate latency improvements. AI complicates that instinct by introducing a genuine trade-off between speed, cost, and quality that doesn't exist in the same way for deterministic systems.

A smaller, cheaper model will typically respond faster and cost less per call, but it may perform noticeably worse on tasks that require careful reasoning or nuanced judgment. A larger model will typically answer better on those same tasks, at the cost of higher latency and higher spend. Neither is universally correct. The right choice depends entirely on what the task requires and what the business is willing to trade off to get it.

This means the goal of production optimization for AI systems is rarely "minimize latency" in isolation. It's closer to something like acceptable quality at acceptable latency and acceptable cost — three variables that pull against each other, with the acceptable threshold for each one differing by use case. A support triage bot answering low-stakes questions might reasonably prioritize speed. A system drafting legal or financial language might reasonably accept slower, more expensive calls in exchange for materially better output. There is no universal formula that resolves this for every product; there is only the discipline of making the trade-off consciously rather than defaulting to whichever axis is easiest to measure.

Cost Becomes a Runtime Signal

Traditional applications have infrastructure costs that scale in relatively predictable, coarse-grained ways — more servers, more storage, more bandwidth. AI usage economics behave differently, because cost can vary significantly from one request to the next, driven by model choice, input size, output length, reasoning effort, the number of model calls a workflow makes, retries, the volume of content retrieved, tool executions, and the size of the context window involved.

Two requests hitting the exact same product feature can therefore have radically different unit economics, and a workflow can remain technically healthy by every infrastructure metric while its cost per successful task quietly deteriorates. This is worth calling out explicitly as its own discipline: cost observability. Not simply watching the total monthly AI bill roll in — that number arrives too late and too aggregated to inform any specific decision — but understanding cost in context: cost per request, cost per conversation, cost per successful task, cost per customer, cost per workflow, cost per model route, cost per resolved support ticket.

No organization needs to track every one of these simultaneously. The point is not to prescribe a universal metric set. The point is that AI cost needs to be connected to useful work, not treated as an undifferentiated line item, because a rising bill and a rising cost-per-successful-outcome are two very different problems that call for two very different responses — one might mean the product is simply succeeding more often and getting more expensive as a natural consequence, and the other means something has gone wrong.

The $0.04 Request and the $4 Workflow

These figures are illustrative, not sourced pricing — but the shape of the problem they illustrate is real. A single model call, priced per token, can look genuinely inexpensive in isolation. An agentic workflow built from several of those calls chained together tells a very different story: retrieve some context, call one model to plan, call another to execute, invoke a tool, retry once because the tool result wasn't quite right, hand off to a second agent for review, generate a long final response. Each individual step might be cheap. The workflow that strings them together might not be.

This is why evaluating AI economics purely from a model's advertised per-token price is a category error for anything beyond the simplest single-call feature. The cost that matters to the business is the cost of the workflow that actually runs in production, including every retry, every fallback, and every unnecessary step — not the sticker price of the model doing the final step. Executives who reason about AI cost only from pricing tables are reasoning about a number their product rarely actually pays.

When Model Routing Becomes an Economic Control System

Many production AI architectures now route requests among several models based on task complexity, latency targets, cost constraints, language, modality, customer tier, or provider availability. This routing layer is genuinely useful — it's how organizations avoid paying frontier-model prices for tasks a smaller model handles perfectly well — but it also introduces a new category of decision that needs its own visibility.

Which model actually handled a given request, and why did the router make that choice? How often does traffic fall through to the expensive fallback path, and is that happening for the reasons the system was designed around, or because something upstream is misbehaving? Has routing behavior itself changed over time, gradually shifting more traffic toward the expensive tier without anyone deciding that should happen? Is the cheaper model, in practice, actually producing acceptable outcomes, or is it quietly generating more customer escalations that cost more downstream than the model saved upstream?

Model routing turns a piece of infrastructure into an economic control system, which means observing it is no longer just an engineering concern — it's directly a finance concern, and the two functions need to be looking at the same evidence.

Retrieval Needs Its Own Visibility

In systems that combine a model with external knowledge — retrieval-augmented generation, in the industry's shorthand — a poor answer frequently has nothing to do with the model at all. The model can be reasoning perfectly well over context that simply isn't the right context.

Important signals here include which documents were retrieved, how they were ranked, how relevant they actually were to the query, how fresh the source material is, what information was missing entirely, retrieval latency, and outright retrieval failures. The core distinction worth internalizing is that model quality and information quality are different problems, measured differently and fixed differently. An excellent model given outdated or irrelevant information will confidently produce an outdated or irrelevant answer — and it will do so with exactly the same fluent, assured tone it uses when it's correct, which is precisely what makes this failure mode dangerous. Nothing about the model's confidence level distinguishes a well-grounded answer from a badly grounded one. Observability has to make the information path itself visible, independent of whatever the model does with that information once it has it.

The Knowledge Pipeline

It helps to think of the path information travels as a pipeline with several distinct stages, each of which can fail independently: a source document is ingested, parsed, broken into chunks, embedded and indexed, retrieved in response to a query, ranked against competing candidates, assembled into context, and finally handed to the model.

A dashboard reporting a near-perfect model API success rate says nothing about whether half of the retrieved documentation is six months out of date, whether the chunking strategy is splitting critical information across boundaries that make it unretrievable, or whether the ranking step is consistently burying the one document that actually answers the question. This is the clearest illustration of why AI reliability depends on more than model reliability — a system can have a flawless model and a broken knowledge pipeline, and every model-level metric will report that everything is fine.

Source Freshness as a Production Metric

Knowledge-intensive AI products carry a form of decay that has no real analogue in traditional software: correctness can degrade without any deployment happening at all, because the world underneath the system keeps changing while the software stands still.

Policies change. Prices change. Documentation gets updated somewhere the system doesn't know to look. Products get discontinued or modified. A knowledge base that was accurate the day it was built becomes gradually less accurate as reality moves on around it, and nothing about the software itself has to change for that to happen. The software didn't change — reality did, and the software simply didn't notice.

This argues for treating source freshness as a genuine production metric rather than a data-hygiene afterthought: how stale is the ingested content, how successfully and how often is it being refreshed, how much delay exists between a real-world change and its reflection in the knowledge base, and how complete is the coverage of the domain the system is expected to answer questions about. None of this requires deep technical machinery to explain to a leadership team — it requires the recognition that a knowledge-grounded AI product has an expiration clock running on its own accuracy, separate from any release cycle.

Agents Turn Traces Into Behavioral Maps

Traditional applications largely execute workflows that engineers explicitly defined in advance. Agents complicate this by dynamically deciding their own sequence of actions at runtime — search the knowledge base, query the customer's account, consult another model, call the billing system, check a policy, retry a failed step, then respond. A different request through the same system might take a completely different path, and that path is not a bug; it's the point of building an agent instead of a fixed workflow.

This is exactly why traces matter more, not less, in agentic systems: they become the only record of what could reasonably be called the system's behavioral topology — the patterns of paths an AI system actually takes through its models, tools, data sources, and sub-agents while completing tasks. This isn't an established industry term so much as a useful way to frame what a trace store full of agent runs actually represents: not a log of events, but a map of how the system tends to move.

The Agent That Gets the Right Answer the Wrong Way

Here is a failure mode that pure outcome-monitoring will never catch, because by definition the outcome looks fine.

An agent completes a task. The user gets the answer they wanted and walks away satisfied. Judged purely on that outcome, the interaction was a success. But the underlying trace tells a different story: the agent called an expensive model when a cheaper one would have sufficed, touched four separate tools when one would have done the job, retried twice for reasons nobody investigated, retrieved a pile of irrelevant documents alongside the relevant one, and in the process exposed more internal data to the model than the task actually required.

If an organization only ever looks at final answers, this workflow looks excellent — arguably one of its better-performing paths, since the user was happy. Operationally, though, it may be expensive, fragile, slow to scale, and carrying more risk exposure than it needs to. This is the strongest argument in the whole discussion for examining process quality alongside outcome quality: a system can be right and still be a problem, and the only way to see that is to look at how it got there, not just where it ended up.

Loops, Dead Ends, and Tool Thrashing

Agents fail in patterns that are distinct enough to deserve their own names. Repeated calls to the same tool without any change in outcome, cycling back and forth between two sub-agents without converging, retrieving the same information again and again, a context window that keeps growing across retries without the task actually advancing — these are recognizable shapes, not random noise.

The most useful label for one of the most common versions of this is tool thrashing: an agent repeatedly invoking tools without meaningfully improving the state of the task it's working on. What makes this pattern particularly easy to miss is that every individual tool call in a thrashing sequence can be a perfect technical success — the tool responded, correctly, every time. HTTP success rates stay pristine throughout. The inefficiency lives entirely at the level of the workflow's shape, not at the level of any single request, which is exactly why infrastructure monitoring alone will never surface it. Useful signals for catching this include tool calls per completed task, detection of repeated call patterns, overall path length, retry counts, and completion rates — not as a mandatory checklist every team must implement, but as the kind of evidence that makes thrashing visible instead of invisible.

Output Quality Needs Evaluation, Not Just Telemetry

Everything discussed so far — traces, latency breakdowns, cost per workflow, retrieval visibility, behavioral topology — answers the question of what happened. None of it, on its own, answers whether what happened was good. That second question belongs to a genuinely different discipline: evaluation.

It's worth being precise about three terms that get used almost interchangeably in casual conversation but mean different things in practice. Monitoring means watching known signals against known thresholds — is latency within range, is the error rate acceptable. Observability means collecting enough evidence about a system's internals to investigate behavior that wasn't anticipated in advance — the trace exists so that when something surprising happens, someone can figure out why. Evaluation means assessing whether a specific piece of behavior or output actually meets a quality bar — not "what happened," but "was what happened acceptable." These three overlap heavily in practice and often live in the same tooling, but they are not the same activity, and conflating them is how organizations end up with excellent traces and no real idea whether their product is any good.

You Cannot Put "Correctness" on a CPU Graph

AI quality is genuinely multi-dimensional, and which dimensions matter depends entirely on the application. Factual correctness, task completion, relevance, instruction-following, policy compliance, groundedness in the retrieved evidence, and user satisfaction are all legitimate quality dimensions — but no product needs to optimize all of them equally, and treating them as a single undifferentiated "quality score" tends to obscure more than it reveals.

A coding assistant and a medical-document summarizer should not share an evaluation rubric. A customer-support system probably cares intensely about policy correctness, resolution rate, and knowing when to escalate to a human rather than guess. A creative writing assistant is optimizing for something almost entirely different. The underlying point is simple to state and easy to skip in practice: AI observability needs application-specific definitions of what good behavior actually looks like. There is no generic quality metric that transfers cleanly across use cases, and organizations that try to force one usually end up measuring something that correlates weakly with what their users actually experience.

Evals Are the Tests That Keep Running After Release

Offline evaluation — running a system against a curated dataset with known correct answers before it ships — plays a role analogous to traditional pre-release testing, and it's a genuinely useful gate. But it cannot be the whole story, because production traffic will always surface scenarios that a curated dataset didn't anticipate.

This is where production evaluation earns its place: sampling real traffic, running it against evaluation criteria continuously, and tracking how behavior shifts over time rather than only checking it once before release. The available techniques span curated regression datasets built specifically to catch known failure modes, ongoing sampling of live production traffic, human review for the cases that matter most, model-based evaluation where it's genuinely appropriate, and direct feedback loops from business outcomes.

Model-based evaluation — using one model to judge another model's output — deserves a specific caveat rather than blanket endorsement. It is useful, increasingly common, and genuinely helpful at scale, but it is not a solved problem. Judge models carry their own biases, can be inconsistent across runs, and need validation against human judgment for the specific task they're being used to evaluate, not just trusted because they're convenient. Evals, in other words, need their own quality assurance. An evaluation system nobody has validated is not meaningfully more trustworthy than no evaluation system at all — it just feels more trustworthy, which may be worse.

The Evaluation Gap

Alongside the Observability Gap sits a related but distinct problem worth naming on its own: the Evaluation Gap.

An organization can build extensive, sophisticated tracing — full visibility into every model call, every tool invocation, every retrieval step — and still have no reliable definition of what "good" means for its product. It can see every step the system took and still not know whether the resulting answer was actually correct. This is a genuinely different failure than the Observability Gap. The Observability Gap is insufficient visibility into what happened. The Evaluation Gap is insufficient ability to judge whether what happened was acceptable, even once you can see it clearly.

The distinction matters because the two gaps close through different work. Closing the Observability Gap means instrumenting more of the system. Closing the Evaluation Gap means doing the harder, more judgment-dependent work of deciding — explicitly, and specific to the use case — what correctness, relevance, and success actually mean for this particular product. Many organizations invest heavily in the first and barely touch the second, which produces the strange and common situation of teams that can watch their AI system fail in exquisite detail without being able to say, with confidence, when it's succeeding.

Traces Without Evals Create Forensics, Not Quality

Tracing is genuinely excellent for investigation — when something goes wrong and someone needs to understand why, a good trace is often the difference between a fifteen-minute diagnosis and a multi-day guessing exercise. But no organization can manually inspect every production trace at any meaningful scale, and tracing alone doesn't change that constraint.

At scale, organizations need ways to detect patterns across traces, sample intelligently rather than reviewing everything, score outputs systematically, cluster similar failures together, compare behavior across versions, and identify regressions before they become widespread. The objective is turning raw traces into actionable evidence rather than an ever-growing archive that nobody has time to read. This is the feedback loop where traces and evaluation genuinely reinforce each other — traces supply the raw material, evaluation turns that material into a judgment, and the judgment feeds back into what gets fixed. It's worth resisting the temptation to over-promise automation here, though. For high-risk use cases in particular, human domain expertise remains essential, and no amount of automated scoring fully replaces someone with real judgment looking at the cases that matter most.

AI Failures Can Be Semantic

A useful category to name explicitly: semantic failure, where a system operates in a technically correct way but produces behavior that is wrong in meaning or business context, with nothing about the technical execution hinting that anything went wrong.

A chatbot might answer a billing question by pulling from the shipping policy — every system involved executed correctly, and the answer is still nonsense in context. An agent might issue a refund of the correct amount to the wrong category of account. A summarization tool might accurately summarize the correct document while omitting the one condition that actually mattered to the reader. A coding assistant might produce syntactically valid, well-structured code that implements a subtly wrong interpretation of the request.

Semantic failures are harder to alert on than technical failures precisely because nothing in the infrastructure layer registers them as failures at all. The infrastructure sees success from start to finish. The customer sees failure. Bridging that gap is not a matter of adding more infrastructure alerts — it requires the evaluation layer discussed above, because semantic correctness is not something status codes were ever designed to represent.

From Error Rate to Bad-Outcome Rate

Traditional error rate is, almost by definition, a measure of technical failure — exceptions, timeouts, non-2xx responses. AI applications benefit from a complementary, harder-to-pin-down measure: something like a bad-outcome rate, tracking how often the system produced a result that was technically fine but substantively wrong.

This will never be perfectly measurable, and it's worth being honest about that rather than pretending otherwise. Reasonable proxies include human corrections, negative feedback, conversation abandonment, escalations to a human, users re-prompting or rephrasing, retried tasks, detected policy violations, and downstream reversals of actions the system took. Every one of these proxies can mislead in specific circumstances — a low correction rate might mean the system is doing well, or it might mean users have simply given up correcting it. No single metric represents AI quality on its own. The value of these proxies is in connecting hard technical telemetry to softer behavioral evidence, not in producing one clean number leadership can watch on a chart.

The User Is Part of the Observability System

Users frequently notice AI quality problems before any engineering metric does, which makes their behavior itself a legitimate signal worth capturing deliberately rather than treated as anecdotal noise.

Useful signals include repeated reformulation of the same question, abandoning a conversation partway through, immediately undoing an action an agent just took, explicitly asking for a human, or copying an answer and later reporting that it turned out to be wrong. It's worth being skeptical of the most obvious feedback mechanism — a simple thumbs-up or thumbs-down — as a sufficient signal on its own. A user can dislike an answer that was actually correct, for reasons that have nothing to do with accuracy. A user can approve an answer that was confidently, fluently wrong, precisely because it sounded convincing. User feedback is evidence to be weighed alongside other signals, not a verdict to be taken at face value.

The Silent Failure Problem

The hardest AI failure to catch is also, unfortunately, one of the most common in practice: the system produces a plausible-sounding answer, the user trusts it without question, nobody complains, and the answer is wrong.

There's no exception thrown. No retry triggered. No negative feedback logged. Often no metric moves in any noticeable way, because nothing about the interaction looked unusual from any system's point of view. This is the clearest possible illustration of why passive monitoring alone is inadequate for consequential AI applications — a failure that produces no signal cannot be caught by any system built only to react to signals. For higher-stakes use cases, this argues directly for active evaluation and deliberate sampling rather than waiting for something to trip an alert, because the whole category of silent failure is, by construction, invisible to alerting. How much of this active evaluation a given system needs should scale with the consequences of getting it wrong, not with how easy the evaluation is to build.

Behavior Can Change Without a Code Deployment

Traditional engineering has a deeply ingrained habit of correlating regressions with releases — something broke, so check what deployed recently. That habit becomes actively misleading with AI systems, because behavior can shift for reasons that have nothing to do with a code change at all: a model provider updates the underlying model, a prompt gets tweaked, routing logic changes, retrieval configuration shifts, new documents get ingested into the knowledge base, a tool's behavior changes on the other end of an API, conversation memory accumulates differently, context construction changes, the mix of what customers are asking about shifts, or a model gets deprecated and silently swapped for a newer one.

The consequence is that "what changed?" becomes a genuinely harder question to answer than it used to be, and answering it requires connecting observed behavior back to versions and configuration across several independent dimensions at once — application version, prompt version, the specific model and its configuration, retrieval configuration, the state of the knowledge base at a given point in time, and tool versions. None of this needs to become an exhaustive implementation exercise for a strategy discussion. The point that matters is causal: without this connective tissue, a genuine regression and an unrelated coincidence look identical, and teams end up debugging the wrong thing.

The Deployment That Never Happened

Picture the scenario without a timestamp or a fictional team attached to it, because the shape of it is what matters. Quality scores start deteriorating. Engineering checks the deployment history and finds nothing recent. Infrastructure health looks completely normal. After more digging than anyone expected to need, the team eventually traces the degradation back to a change nobody deployed — an external knowledge source that updated on its own schedule, or a model route that shifted upstream without any internal change triggering it.

This is a new kind of operational reality that traditional engineering intuition doesn't prepare anyone for: AI regressions do not reliably correspond to application deployments. The natural first move — "check what we shipped" — can waste real time in exactly the cases where the actual cause was never under the team's direct control to begin with.

Drift Is Not One Thing

"Drift" gets used as a catch-all term in AI discussions, which makes it nearly useless as a diagnostic category unless it gets broken apart. It's worth distinguishing at least a handful of genuinely different phenomena that all get called drift.

Data drift is a change in the inputs a system receives — the questions users ask, the documents fed into it, the shape of the traffic itself. Knowledge drift is a change in the information required to answer correctly, even when the system itself hasn't changed at all — this is the source-freshness problem discussed earlier, viewed from a different angle. Behavioral drift is a shift in the system's outputs or decision patterns — the same inputs start producing different outputs over time. Cost drift is the same workflow gradually becoming more expensive without a corresponding change in what it delivers. Usage drift is customers beginning to use the system in ways its original design never anticipated, stretching it into territory it wasn't built or evaluated for.

These categories overlap in practice — usage drift can trigger behavioral drift, knowledge drift can look like behavioral drift from the outside — but naming them separately matters, because the fix for one rarely fixes another. A team that diagnoses cost drift as behavioral drift will spend its effort in the wrong place. Defining which kind of change an organization actually cares about, for a given system, has to come before choosing what to measure.

Model Upgrades Become Production Experiments

Switching to a new model version often looks like an obviously good idea on paper — cheaper, faster, more capable on public benchmarks. What that framing leaves out is that replacing an underlying model can alter far more than the headline capability numbers suggest: tone, reasoning style, tool-selection behavior, the reliability of structured output, how closely instructions get followed, token consumption per response, and latency all tend to shift together, sometimes in surprising directions.

This is why model upgrades deserve to be treated as production experiments rather than routine swaps, evaluated against the application's actual objectives rather than assumed safe because a general benchmark improved. In practice this tends to involve some combination of side-by-side evaluation against the previous model, canary traffic that limits exposure while confidence builds, shadow evaluation that runs the new model in parallel without serving its output, controlled A/B testing where the product supports it, and regression datasets built from real production cases the old model was known to handle correctly. No single method is the correct one for every situation — the underlying principle that holds across all of them is that model quality has to be demonstrated inside the specific application, not assumed from how a model performed somewhere else entirely.

Benchmarks Do Not Run Your Business

A model can lead the public leaderboards and still underperform, specifically, for a company's own support tickets, its particular domain vocabulary, its specific tool architecture, its latency requirements, and its cost constraints. None of that is a contradiction — benchmarks measure something real, just not the thing a specific business actually needs measured.

Benchmarks remain useful for a first-pass sense of a model's general capability. But they answer a different question than the one that determines whether a model swap will actually help a given product. Production observability answers the question benchmarks structurally can't: how does this model behave inside this particular system, with this data, these users, this tool architecture, and these constraints. That's a question no public leaderboard was designed to answer, and no amount of general benchmark superiority substitutes for actually finding out.

AI Observability Is a Product Discipline, Not Just an SRE Tool

Everything covered so far could sound like it belongs entirely to engineering. It doesn't. AI behavior has consequences that ripple across product, QA, data, security, support, and finance, and each of those functions ends up asking a version of the same underlying question from its own vantage point.

Engineering asks why a specific workflow failed. QA asks whether a regression is reproducible and under what conditions. Product asks whether users are actually completing the goals the feature was built for. Finance asks why cost per conversation is rising when usage hasn't grown proportionally. Security asks what data an agent actually accessed while completing a task, and whether that access was appropriate. Support asks why customers keep asking to be transferred to a human. Leadership asks, more bluntly than any of the above, whether the AI feature is creating enough value to justify what it costs to run.

The encouraging part is that the same underlying observability infrastructure can support all of these questions at once — it doesn't require a separate system per function. What it does require is recognizing, from the start, that the infrastructure is being built for more audiences than the engineering team that instruments it.

The Dashboard Needs Multiple Audiences

It follows that one enormous, all-purpose AI dashboard is unlikely to satisfy anyone well, because the roles asking these questions genuinely need different views into the same underlying evidence. SRE teams care most about availability, latency, and provider-level failures. AI engineers care most about traces, model calls, retrieval behavior, and tool use. QA cares about regression patterns, evaluation scores, and clusters of failure. Product cares about task completion, adoption, and where users abandon a flow. Finance cares about unit economics.

None of these views is more correct than the others — they're different lenses on the same underlying system. The important shift this implies is that observability stops being a purely engineering artifact and becomes shared evidence across functions, which has real organizational consequences: it means the trace store, the eval results, and the cost data need to be legible and accessible to people who aren't engineers, not locked inside a tool only the SRE team knows how to query.

The AI Observability Question Tree

It's useful to pull the progression covered so far into a single framework — not a generic seven-layer stack borrowed from a vendor slide deck, but organized the way an organization would actually walk through it when something looks wrong.

Is it running? This is the infrastructure-health question, and it still has to be answered first.

Is it responding normally? Latency, provider behavior, and error rates one layer up from raw availability.

What path did it take? The trace itself — routing decisions, model calls, tools invoked, retrieval performed.

What information influenced it? The context, retrieved documents, memory, and tool outputs the model actually had in front of it.

Was the behavior acceptable? This is where evaluation enters — correctness, policy compliance, task success.

Is the behavior changing? Regression detection and drift, tied back to specific changes in model, prompt, or configuration.

Is the economics changing? Tokens, model selection, retries, and cost measured against successful outcomes rather than in isolation.

Did the user get value? The business and user-outcome question that everything above ultimately exists to answer.

This works as a diagnostic sequence more than a mandatory checklist — not every organization needs equal depth at every layer for every system, and the right depth depends on the risk and complexity of what's being built. But the sequence itself is a useful discipline: when something looks wrong, working down through these questions in order tends to surface the actual cause faster than jumping straight to a guess.

The Three Truths of Production AI

A second, smaller framework is worth holding alongside the question tree, because it captures something the question tree doesn't quite say directly.

Operational Truth is what happened technically — the infrastructure-level record of execution. Behavioral Truth is what the AI actually did — the trace, the path, the decisions along the way. Outcome Truth is whether that behavior actually achieved an acceptable result.

An organization can possess Operational Truth without Behavioral Truth — perfect infrastructure logs and no idea what the agent actually did along the way. It can possess Behavioral Truth without Outcome Truth — a complete trace of every step, with no way to say whether the result was good. Mature AI observability is the ongoing work of connecting all three, because any one or two of them in isolation leaves a real blind spot exactly where the organization needs the most confidence.

Observability Without Action Is Expensive Storage

None of the preceding sections matter if they don't eventually produce a decision. Collecting enormous volumes of traces, prompts, outputs, evaluation scores, and token metrics does not, by itself, create reliability — it creates a very large and very expensive archive.

Teams need the collected evidence to lead somewhere: investigate a specific failure, roll back a prompt change, adjust routing logic, fix a retrieval configuration, block a tool that's being misused, update an evaluation dataset to catch a newly discovered failure mode, or escalate a case to human review. The actual purpose of observability is not accumulating data. It's reducing the time between unexpected behavior occurring and the organization understanding it well enough to act — and an observability system that produces beautiful dashboards nobody acts on has, in practice, failed at its one real job.

AI Alert Fatigue

There's a specific failure mode worth naming on the way to good observability: building so many AI-specific metrics and thresholds that the resulting system produces less understanding rather than more. A dashboard tracking two hundred metrics, generating eighty possible alerts, and surfacing a handful of quality scores can genuinely overwhelm a team faster than it informs one.

Quality metrics in particular tend to fluctuate naturally in ways that don't warrant treating every small movement as an incident — a slight dip in an evaluation score from one week to the next is often noise, not signal. Useful counters to this include establishing real baselines before setting thresholds, choosing thresholds that reflect genuinely meaningful change rather than arbitrary round numbers, watching trends over time rather than reacting to single data points, building alerts specific to a segment rather than one blunt organization-wide trigger, applying anomaly detection where it's actually warranted, and keeping human review in the loop for the judgment calls that automated thresholds can't make well. The goal throughout is signal quality over metric volume — a small number of alerts a team trusts and acts on beats a large number they've learned to ignore.

Observability Cardinality Can Explode

AI telemetry tends to produce far more dimensionality than traditional infrastructure metrics ever did — model, prompt version, user, agent, tool, customer, task type, retrieval source, and model route can all be dimensions worth slicing by, and the combination of all of them can grow extremely large extremely fast.

This has real consequences: observability systems built without attention to this can become expensive to run, slow to query, and genuinely difficult to interpret once the dimensionality gets high enough that no human can hold the full picture in mind. This isn't a call for a specific implementation pattern — it's a reminder that telemetry design itself is a design decision with real trade-offs, not a default setting. Observability has an operating cost of its own, in both infrastructure spend and cognitive load, and organizations have to decide deliberately which visibility is actually worth paying for rather than instrumenting everything because it's technically possible.

Privacy and Observability Can Collide

It's worth returning to the privacy question raised earlier, because it deserves more than a passing mention. A production trace can contain customer messages, internal documents, business records, model outputs, and the arguments passed into tool calls — which makes AI observability data, in aggregate, potentially far more sensitive than the metrics traditional infrastructure monitoring ever collected.

This raises questions that don't have universal answers: who inside the organization can actually inspect raw traces, what should be redacted before storage, how long content should be retained, whether customer content can be sampled for evaluation purposes, which environments are allowed to store full prompts versus metadata only, and which regulations apply to a given product's specific data. None of these resolve into one-size-fits-all guidance — the right answer depends on the industry, the jurisdiction, and what the product actually handles, and getting it right requires involving legal and security functions directly rather than treating it as a purely technical decision. The underlying trade-off is unavoidable: debuggability generally improves with more retained content, and data minimization generally improves with less. Neither side of that trade-off should win by default.

Security Needs the Trace Too

AI observability and AI security overlap more than either discipline's name suggests, without being the same discipline. Anomalous agent behavior — unusual tool selection, unexpected data access outside a task's normal scope, an abnormally long sequence of steps, repeated failed actions, or behavior that looks like it was influenced by instructions embedded in untrusted data the agent processed — is exactly the kind of thing a behavioral trace can surface, and exactly the kind of thing a security team needs to investigate when it happens.

Security teams increasingly use the same behavioral traces that engineering and QA use for debugging, just asking a different question of them. Observability, quality, and security begin to overlap in the underlying evidence they draw on, even while their responsibilities and the actions they take in response remain conceptually distinct.

From Monitoring Models to Monitoring Systems

There's a persistent trap worth naming directly: model-centric thinking, where a team becomes so focused on model accuracy that it misses failures happening everywhere else in the system around it. A production AI application is not just a model — it's a model plus orchestration logic, prompts, retrieval, external services, tools, databases, user context, and business rules, all interacting.

A team can obsess over model accuracy while a retrieval pipeline is quietly broken underneath it. Or worse, a team can replace the model entirely — an expensive, disruptive change — when the actual issue was stale data, a prompt regression, tool latency, or flawed routing logic that had nothing to do with the model at all. The unit of observability that matters most, increasingly, is the AI application or workflow as a whole, not the model in isolation. The model is the most visible, most discussed component — similar to an engine inside a much larger vehicle. Knowing the engine's temperature tells you nothing about whether the vehicle actually reached the right destination. The analogy shouldn't be stretched further than that, but the underlying point holds: model monitoring alone is necessary and nowhere near sufficient.

Observability Must Follow the Customer Journey

Individual model calls can each look perfectly healthy while the experience built from them fails. Customers don't experience isolated calls — they experience conversations, workflows, and tasks that may span many calls strung together, and optimizing each call independently doesn't guarantee the overall experience improves.

An AI support interaction spanning twelve separate model calls can have every one of those calls individually perform within acceptable bounds while the conversation as a whole frustrates the customer. Useful session-level questions include whether the customer actually accomplished their goal, how many turns the interaction took, whether the agent repeated itself, whether a human escalation was ultimately required, how expensive the full interaction turned out to be, and where in the flow users tend to abandon. This is journey observability — understanding AI performance at the level of the user's actual task rather than at the level of isolated model requests. The request, as a unit of measurement, is simply too small for systems people experience as ongoing interactions rather than single exchanges. Ten API calls can each succeed individually while the agent never actually completes what the customer came to do, and no per-call metric will ever surface that gap.

Business Metrics Close the Loop

The point of all this observability infrastructure was never beautiful traces, exhaustive logs, or minimized token counts for their own sake. It was always a reliable, economically sustainable product, which means the technical and behavioral signals discussed throughout this piece eventually need to connect back to outcomes the business actually cares about.

For a customer-support AI, that might mean successful resolution rate, escalation frequency, how often a resolved issue reopens, satisfaction, and cost per resolution. For a sales AI, qualified-opportunity rate, the frequency of incorrect claims made to prospects, meeting conversion, and complaint volume. For a developer assistant, the rate at which suggested changes get accepted, any resulting defect or rework, time actually saved, and the qualitative developer experience. None of these should be read as proof of causation just because a metric moves in the right direction at the same time as an AI feature ships — correlation in this space is easy to find and easy to over-read, and it's worth resisting the temptation to claim more certainty than the data actually supports.

Cheaper Is Not Better If Users Need Three Attempts

A short but important caution: a routing change that reduces cost per model call can still make the product more expensive overall, if it pushes users toward needing to repeat their requests more often to get a satisfactory answer. The total cost of a workflow, including the human time and repeated attempts it now requires, can rise even while the isolated per-call cost falls.

The same dynamic runs in the other direction. A faster model might cut latency while quietly increasing escalation rates. A more capable, more expensive model might raise the cost of each individual call while reducing retries enough to lower the total cost of getting to a successful outcome. This is the clearest possible argument for measuring unit economics around outcomes rather than around isolated model requests — a cheaper individual step is not automatically a cheaper journey, and treating it as one is how well-intentioned cost optimization ends up making a product worse and more expensive at the same time.

QA and Observability Begin to Converge

This is where the boundary between testing and production observability genuinely starts to blur, and it's worth being direct about that rather than treating it as a footnote. Traditional QA largely asks, before release, whether a given change behaves correctly. Observability largely asks, after release, what is actually happening in production. AI systems make the line between those two questions much less clean than it used to be.

Production traces can become regression examples, evaluation datasets, useful edge cases, and documented failure scenarios in their own right. Test suites, in turn, can start functioning like production monitors — quality checks and evaluation criteria that run continuously rather than once before a release. This is the Production-to-Test Loop: production reveals unexpected behavior, that behavior becomes a reproducible evaluation or test case, engineering changes the system in response, and the new evaluation prevents the same failure from quietly returning later. This feedback loop matters more for probabilistic systems than it ever did for deterministic ones, because a deterministic bug, once fixed, tends to stay fixed — a probabilistic system's failure mode can resurface under a slightly different prompt, a slightly different model version, or a slightly different mix of user requests, even after the "same" problem was supposedly addressed once already.

Every Bad Trace Can Become a Future Test

Not literally every single trace, but the valuable ones — genuine production failures — can and should feed directly into regression datasets, automated evaluations, targeted testing for specific scenarios, prompt-level tests, tool-selection tests, and retrieval-specific tests. This is what connects observability to continuous improvement in a durable way rather than a one-off fix.

A mature organization doesn't stop at patching individual AI failures as they surface. It converts them systematically into new evidence about what the system needs to handle going forward, which is a meaningfully different and more durable practice than simply reacting to each failure as an isolated incident.

QA No Longer Ends at Deployment

Because AI behavior can shift for reasons entirely outside a code deployment — a model update, a knowledge-base change, a shift in how customers are using the product — quality engineering has to become a more continuous activity than the traditional pre-release testing model assumed.

In practice this tends to look like a genuine collaboration across roles: QA identifies the scenarios that matter most, engineering instruments the system so those scenarios are traceable, production supplies the real edge cases nobody thought to test for in advance, product defines what an acceptable outcome actually looks like, domain experts label the genuinely ambiguous cases that automated scoring can't resolve on its own, and evaluation systems track regressions as they emerge. None of this requires every QA engineer to become a site reliability engineer or a machine learning specialist. It means the historical boundaries between testing, evaluation, and observability become more connected than they used to be, because the systems being tested no longer hold still between releases the way they once did.

SLOs for AI Need Careful Thinking

Traditional service-level objectives are typically built around availability and latency — measurable, largely objective, and reasonably stable over time. AI products understandably want the same discipline applied to quality, but quality is a harder thing to pin a number on, and the difficulty is worth taking seriously rather than papering over.

Possible quality-oriented objectives include successful task-completion rate, a maximum acceptable escalation rate, policy-compliance rate, and grounded-answer rate — how often the system's claims are actually supported by the evidence it retrieved. There is no universal, off-the-shelf "AI SLO framework" that transfers cleanly across organizations, and claiming otherwise oversells what's actually achievable. The genuine challenges include evaluation noise that makes small movements hard to interpret, incomplete or inconsistent labels, a changing mix of users and use cases that shifts the baseline out from under the metric, and quality definitions that are inherently domain-specific. Quality SLOs can be genuinely useful, but only once the underlying measurement has been carefully validated — an SLO built on a shaky metric is worse than no SLO at all, because it creates false confidence.

The Danger of a Single AI Quality Score

There's an understandable pull toward wanting one clean number — "AI Quality: 94 out of 100" — that leadership can watch move over time the way they'd watch an uptime percentage. That number is genuinely attractive to report. It's also genuinely capable of hiding exactly the variation that matters most.

Overall quality can look strong in aggregate while performance is poor for one specific language, for enterprise customers specifically, for one particular intent, for unusually long conversations, or for interactions that involve an unusual tool. Averages are very good at hiding pockets of high-risk failure, precisely because they're designed to smooth variation out rather than expose it. Segmentation is the direct answer to this — deliberately breaking a single quality score apart by the dimensions that matter, rather than trusting the blended number to represent the whole system fairly.

Observe by Segment, Not Only by Average

Useful segmentation dimensions include customer tier, language, use case, the specific model or workflow involved, the specific tool being used, prompt version, and conversation length — without extending into collecting sensitive demographic attributes that aren't actually necessary for the analysis and that raise their own privacy concerns for no proportionate benefit.

The underlying idea is straightforward: AI failures tend to concentrate in specific contexts rather than distributing evenly across all usage, and a system-wide average is structurally built to conceal exactly that kind of concentration. An organization that only ever looks at its blended quality number will consistently be the last to notice that one entire customer segment or one entire use case is being poorly served, right up until it shows up as churn.

Observability Becomes Part of Release Engineering

A release for an AI product frequently bundles together more than code — it can include a new prompt, a new model, a changed routing policy, an updated retrieval configuration, and a modified tool schema, all shipped as though they were one atomic unit even though each carries its own risk of regression.

This argues for teams being able to compare behavior before and after a release across the relevant quality, cost, latency, and behavioral signals — not just confirming the code deployed successfully, but confirming the product still behaves the way it's supposed to once that deployment has taken effect. This is part of why controlled rollouts and canary deployments matter more for AI changes than they might for a simple code change with predictable, deterministic effects. There's no single correct process every organization must adopt here — the principle that matters is comparability across releases, not any particular deployment mechanism.

The Four-Axis Release Check

One compact way to hold this together before expanding any AI change broadly: examine movement along four axes at once rather than optimizing any single one in isolation.

Quality — did behavior actually improve, stay the same, or regress? Reliability — did errors, loops, or workflow failures change in frequency? Performance — did latency or total completion time move, and in which direction? Economics — did cost per successful outcome change, not just cost per call?

The point worth stressing is that a change can genuinely improve one of these axes while quietly harming another — a cheaper model that saves money on paper while degrading quality enough to raise escalations, or a faster response that arrives more quickly but less reliably. Production decisions for AI systems almost always involve trade-offs across these four axes rather than a clean win on every dimension at once, and treating the check as a simple pass/fail gate misses the trade-off entirely.

The Observability Data Itself Needs Governance

The telemetry, traces, and evaluation results an organization accumulates become their own asset requiring its own governance — questions of data retention, who has access to what telemetry, the risk of vendor lock-in, whether traces can be moved between tools, and whether the schema stays consistent enough over time to remain queryable.

This is where standardization efforts become strategically relevant, and it's worth being precise and current about where those efforts actually stand rather than repeating a stale summary. OpenTelemetry's GenAI semantic conventions — a shared vocabulary of attribute names, span structures, and metrics meant to make an LLM call look like an LLM call regardless of which framework or vendor emitted it — have been under active development by OpenTelemetry's GenAI Special Interest Group since 2024. As of mid-2026, these conventions remain formally in "Development" status rather than stable, and in June 2026 they were reorganized out of OpenTelemetry's main semantic-conventions repository into a dedicated repository of their own, reflecting how quickly this particular area is still moving relative to more settled parts of the OpenTelemetry specification. No 1.0 release has shipped as of this writing, and attribute names have continued to evolve across versions.

None of that should be read as a reason to wait before adopting them. The practical posture most engineering teams appear to be converging on is adopting the conventions now, since the overall shape is already reasonably good and widely emitted across tooling, while pinning a specific convention version rather than assuming stability, and expecting some continued change as the specification matures. Framework and platform support has also been growing quickly — several major observability platforms already map OpenTelemetry's GenAI attributes into their own product interfaces.

The Open Standard Question

The strategic case for a CTO to care about this, independent of the implementation details, is straightforward. If every AI framework and every observability vendor emits telemetry in its own proprietary shape, an organization ends up locked into whichever tooling it happened to instrument against first, with real switching costs if that tooling stops fitting the organization's needs. A shared vocabulary for model calls, agent spans, and tool-call representation reduces that fragmentation and preserves the option to change observability vendors without re-instrumenting an entire application from scratch.

The honest caveat is that this convergence is still in progress, not finished. Treating "OpenTelemetry-compatible" as though it already guarantees one consistent telemetry schema across every framework and vendor would overstate where things actually stand today. The direction is sound and worth building toward. The maturity isn't there yet, and pretending otherwise sets teams up for schema surprises down the line.

The AI Observability Market Will Consolidate and Fragment at the Same Time

The tooling landscape around AI observability is genuinely crowded, and it's changing quickly enough that any specific product comparison risks going stale within months of being written. Broadly, the categories at play include traditional application-performance-monitoring vendors extending existing products with AI-specific visibility, dedicated LLM observability platforms built AI-native from the start, agent frameworks that ship with their own built-in tracing, cloud-provider tooling bundled into existing platforms, open-source projects, and evaluation-focused platforms that overlap heavily with observability without being identical to it.

The boundaries between monitoring, observability, evaluation, prompt management, and experimentation have become genuinely blurry across this landscape — many platforms now touch several of these categories at once rather than specializing narrowly in one. That blurring is likely to continue rather than resolve cleanly, which makes predicting specific winners a poor use of a strategy discussion. What's durable is the set of architectural concepts this piece has walked through — tracing, evaluation, cost-per-outcome, drift, journey-level visibility — regardless of which specific vendor ends up implementing them best in any given year.

Build vs. Buy Is the Wrong First Question

A natural instinct, once an organization recognizes it has an observability gap, is to ask which platform to purchase. That question, asked first, tends to produce a poor answer, because tools cannot compensate for unclear thinking about what quality actually means for a specific product.

Before evaluating any platform, an organization genuinely needs to have worked through what failure looks like for its specific use case, which workflows actually matter enough to warrant deep instrumentation, what needs to be traceable versus what can reasonably be left uncaptured, what quality means for this particular product in concrete terms, what data can and cannot be stored given privacy and compliance constraints, and which business outcomes the whole effort is ultimately meant to serve. This is analogous to purchasing analytics software before deciding what the business actually wants to measure — the tool will happily collect data in whatever shape you configure it to, and a poorly configured tool produces an expensive illusion of visibility rather than actual visibility.

Observability Maturity Is Really Question Maturity

The most useful way to think about organizational maturity in this space isn't how much telemetry an organization collects — it's the sophistication of the questions it's actually capable of answering.

An immature organization asks whether the API is up. A more advanced one asks why users seem dissatisfied. A more mature one asks which specific execution patterns correlate with unsuccessful outcomes. A genuinely mature one asks which proposed change will improve quality without making latency or unit economics unacceptable — holding all three in view at once rather than optimizing one in isolation. Observability maturity, understood this way, is really about improving the questions an organization is equipped to answer, which matters considerably more than the sheer volume of telemetry it happens to be collecting. An organization drowning in dashboards that still can't answer the third or fourth question on that list has not solved its observability problem — it has just made the problem more expensive to store.

The Observability Maturity Curve

It helps to lay this progression out as a curve of its own, described in plain terms rather than the generic numbered levels that get recycled across most maturity-model diagrams.

See the Machine — basic infrastructure health: is the system up, is it responding, are errors within normal range.

See the Call — visibility into individual model requests: which model, what latency, how many tokens.

See the Path — full tracing across retrieval, tools, and agent steps, not just the final model call.

See the Behavior — evaluation and quality signals layered on top of the trace, answering whether what happened was actually good.

See the Change — regression detection and drift, tied back to specific causes across model, prompt, and configuration changes.

See the Economics — cost connected to workflow outcomes rather than tracked in isolation.

See the Customer — business and user impact, the layer everything below ultimately exists to serve.

Not every system needs equal depth at every layer, and that's an important qualifier rather than a footnote. A low-risk internal summarization tool can reasonably operate with much shallower observability than a system making financial or medical decisions on a customer's behalf. Risk and consequence should determine how deep an organization needs to go, not a one-size-fits-all target maturity level applied uniformly across every AI feature a company builds.

A Low-Risk Copilot and a Financial Agent Should Not Have the Same Observability Requirements

This is worth stating plainly rather than leaving implicit. A low-impact internal tool — an assistant that summarizes meeting notes, say — can reasonably tolerate relatively simple monitoring, because the consequences of an occasional bad output are genuinely limited.

An autonomous system making decisions with financial, security, medical, legal, or customer-account consequences needs meaningfully deeper traceability, more rigorous evaluation, more human review built into its workflow, and stronger auditability, because the cost of a silent failure in that context is categorically different. None of this is a legal claim about specific regulatory requirements, which vary by jurisdiction and industry and deserve their own dedicated research rather than a general assertion here. The organizing principle that holds regardless of jurisdiction is simpler: observability investment should scale with consequence, not with how technically interesting a given system happens to be to instrument.

AI Observability Is How AI Becomes Engineerable

Software engineering became reliable, in large part, because developers gained real visibility into what their production systems were actually doing. Distributed systems once seemed genuinely opaque — a request disappeared into a mesh of services and either a correct answer came back or it didn't, with little insight into what happened in between. Modern observability made those systems dramatically easier to debug, optimize, and operate, not by making them simpler, but by making their behavior visible enough to reason about.

AI applications present a new version of that same underlying visibility problem, one level up. Teams now need to understand systems that contain genuinely probabilistic behavior — components that can produce different, defensible outputs from the same input, and whose correctness can't be verified by re-running the same test twice. Observability doesn't make AI deterministic, and it shouldn't be sold as though it could. What it does is make the uncertainty that's inherent to these systems considerably more manageable — turning "we don't know why it did that" into "here's exactly what it did, and here's why."

AI Observability Is Not About Reading the Model's Mind

It's worth being explicit about what this whole discipline does not claim to do. None of the tracing, evaluation, or telemetry described throughout this piece provides genuine access to a model's internal reasoning process in any deep, interpretability sense. Claiming that traces let an organization see "why" a model reasoned a certain way, in a mechanistic sense, overstates what's actually being observed.

What's genuinely observable — and genuinely valuable — is behavior: the inputs a system received, the outputs it produced, the tools it called, the information it retrieved, the model it used, how long each step took, what it cost, the path the workflow took through the system, and how evaluation scored the result. That's not philosophical interpretability, and it doesn't need to be. It's production engineering, aimed at a practical and achievable goal rather than a much harder scientific one.

The New Control Room

Return, finally, to the green dashboard this piece opened with. A modern AI control room isn't one literal giant screen replacing the old one — it's a connected set of views spanning infrastructure health, execution traces, model usage, retrieval behavior, agent behavior, evaluation results, cost, and customer outcomes, each serving a different audience while drawing on shared underlying evidence.

Traditional dashboards were built to answer one question well: is the machine running? That question still has to be answered, every day, without exception. But it is no longer sufficient on its own, and the new observability layer described throughout this piece exists to answer the question sitting right behind it: is the machine doing useful work? That second question is the actual destination all of this has been building toward.

Final Argument

The title of this piece — you can't manage AI software with yesterday's dashboards — was never meant to suggest that traditional dashboards are obsolete. They aren't. It means something more specific: yesterday's dashboards were built around systems where successful execution provided strong evidence of successful behavior, because for most of software's history, that coupling was tight enough to rely on.

AI weakens that coupling significantly. A modern AI system can be available, fast, and technically error-free while simultaneously becoming less accurate, less useful, more expensive, and harder to predict — and every one of those trends can run for weeks without tripping a single conventional alert. That is not a hypothetical edge case. It is close to the default trajectory for any AI product that isn't being deliberately watched at the behavioral level.

This is why production AI management requires a genuinely new layer of visibility, not a replacement for the old one. The strongest organizations working in this space will not distinguish themselves simply by shipping more AI features faster than their competitors. They will distinguish themselves by their capacity to answer a specific set of questions with real confidence: what happened, what path produced it, whether it was acceptable, whether the behavior is changing over time, what it actually cost, and whether the person on the other end of the interaction got something of value from it. Infrastructure reliability remains non-negotiable — it always was and it still is. But behind every green light on a modern AI dashboard now sits a further question that light was never designed to answer, and the organizations that build the capacity to answer it will be the ones able to improve their AI systems faster and more safely than the ones who can only tell whether their APIs stayed online.

Visibility, in the end, is what makes it possible to change an AI system with confidence instead of hope.

---

Sources and Further Reading

• OpenTelemetry, "Semantic Conventions for Generative AI Systems" and related GenAI SIG documentation, including the June 2026 reorganization into the dedicated semantic-conventions-genai repository (opentelemetry.io and the OpenTelemetry GitHub organization).
• OpenTelemetry blog, "Inside the LLM Call: GenAI Observability with OpenTelemetry" (opentelemetry.io/blog).
• LangChain, "State of Agent Engineering" report, based on a survey of over 1,300 professionals conducted November–December 2025 (langchain.com).
• CNCF and OpenTelemetry project documentation on distributed tracing, metrics, and semantic conventions generally.
• Industry reporting and technical analysis on LLM and agent observability platforms and market trends, used here only where clearly attributed as vendor or third-party research rather than independent consensus.

Figures on cost, pricing, and adoption cited in this article are either explicitly labeled as illustrative or attributed to a specific named source and survey context; readers evaluating vendor-specific pricing or capability claims should verify current figures directly with the relevant provider, as this is an area that continues to change quickly.