Anyone who’s ever been on an on-call rotation knows the feeling. A dashboard lights up green to red across a dozen services, an incident channel starts filling with theories and somewhere in that noise is one root cause hiding behind six symptoms that all look equally suspicious. Fixing the problem is rarely the hard part. Figuring out what the problem actually is- that’s where the time is spent.

So, when yet another vendor pitch promises that AI has solved this (point a model at your telemetry and it tells you what broke), part of me wants to believe it. Honestly, some of that promise is real. However, the more time you spend around incident response, the easier it gets to spot claims that skip past the hardest part of the job. Finding symptoms is easy; understanding causes is not. This is just my opinion on where that gap actually sits, and what it would really take to close it.

The Gap Between Correlation and Diagnosis

Look closely at what most ‘AI-driven root cause analysis’ tools actually do, and a pattern starts to show up. They’re very good at telling you that 47 alerts are all part of the same problem. They’re much less reliable at telling you what that problem actually is. Alert correlation, topology mapping, and noise reduction are genuinely useful capabilities that meaningfully reduce the time an engineer spends on triaging. However, grouping related symptoms isn’t the same thing as identifying a cause.

That distinction isn’t just semantics. It’s the difference between an engineer opening an incident channel to “here are the 12 things that fired around the same time” versus “here’s why they fired, in that order, and here’s what to fix.” The first still needs a human to do the actual diagnostic reasoning. The second means the system has built something closer to a real causal model, an understanding of how a change three services upstream produced a symptom two hops downstream, three minutes later.

Most tools on the market today are much closer to the first category than they’d like you to believe.

Sketching What Real Diagnosis Looks Like

If correlation is “these things happened together,” diagnosis is “this happened because of that.” Getting from one to the other isn’t just a matter of bolting on more dashboards. It’s a genuinely different process, and I think it’s worth actually sketching out instead of leaving it vague.

At a high level, a system capable of real causal diagnosis needs to move through like this:

The step that quietly gets skipped is testing causal direction. Does perturbing the upstream service actually explain the downstream symptom, or are the two just moving together? Most tools stop at correlation dressed up as causation, because that step is genuinely hard to build. Matching against incident history is where organizational discipline comes into play, since that knowledge base doesn’t build itself. The confidence check at the end, the part where the system is willing to say, “I’m not sure, escalate this,” is the one I’d trust least to actually exist in most tools being sold today. Uncertainty doesn’t make for a great product demo.

Where This Gets Harder: Non-Deterministic Systems

All of this is already hard for traditional infrastructure, where failure modes are at least somewhat bounded and repeatable. It gets a lot harder once AI and LLM-based components are part of the system you’re trying to diagnose.

The traditional signals SRE teams have leaned on for a decade (latency, traffic, errors, saturation), all assume a system that behaves the same way twice given the same inputs. A RAG pipeline or an agentic workflow doesn’t offer that guarantee. The same request can fail differently on different days for reasons that have nothing to do with infrastructure health: A shift in retrieved context, a subtle change in model behavior and a guardrail stepping in somewhere unexpected. A diagnostic system trained to reason about deterministic failure patterns will confidently give you the wrong answer here, and it’ll say it with the same tone of authority it uses when it’s right.

That’s the part that worries me most, honestly, more as a leader than as an engineer. An automated system that’s occasionally, confidently wrong isn’t neutral. It actively erodes an on-call engineer’s willingness to trust it, often for good, after just one bad call. Trust, once a team loses it in a tool, is brutally hard to earn back. It’s not uncommon to see decent tooling get quietly abandoned after exactly this kind of incident, and it rarely comes back into rotation once that happens.

The Human-in-the-Loop Argument, Restated

None of this is an argument against automating incident diagnosis. The toil is real, the pages at inconvenient hours are real and the case for AI assistance in incident response is a strong one. It’s an argument for being precise about what’s actually being automated, and for treating the postmortem, not the dashboard, as the real training ground for these systems.

A good postmortem is a structured record of a causal chain that a human worked out under pressure. That’s exactly the kind of data the ‘match incident history’ step in the flow above needs to work well. This is where leadership matters just as much as architecture. A team that treats postmortems like a checkbox exercise, written thin and fast just to close out a ticket, is quietly starving its own diagnostic tooling of the one input that would actually make it smarter. A team that puts real-time into writing specific, causally clear postmortems, even when it’s tempting to just write “flaky network” and move on, is building the training data for tomorrow’s automation. Whether or not anyone thinks of it that way in the moment.

This is one of the more underrated jobs of an engineering leader right now. It’s not about picking the right AI vendor; it’s about making sure a team’s own incident discipline is good enough to be worth automating in the first place.

A Practical Checklist

Before adopting or building automated incident diagnosis, a few questions are worth asking honestly:

• Does this tool group related alerts, or does it actually explain why one caused another?
• Can it show its reasoning in a way an engineer could defend in a postmortem review?
• Does it work off the live dependency graph, or a stale architecture diagram from months ago?
• How does it hold up on non-deterministic components, and does it know when to say “I’m not confident” instead of guessing?
• Is there a feedback loop where past incidents actually improve future diagnosis, or does that knowledge just sit locked in a wiki page?

The teams that get real value out of automated diagnosis won’t be the ones with the fanciest model. They’ll be the ones who were already disciplined about turning incidents into institutional knowledge, and who are now feeding that discipline into the machine instead of hoping the machine invents it from scratch. That’s a leadership problem before it’s an engineering one, and I think it’s the one most of the current conversation keeps skipping past.