Confidence Scores Are Closed Loops

When you ask an AI agent how sure it is, you often get something like “I’m 85% confident this is correct.”

This feels useful. It’s not. Here’s why, and what actually works.

The Number Refers to Nothing

What’s the difference between 83% and 87% confident? How was it calibrated? Against what dataset? What does it even mean for a language model to assign probability to its own correctness?

The number terminates in the model’s internal state. There’s no external reference. You can’t verify it, can’t act on it, can’t use it to decide what to check next.

This is what I call a closed loop - a claim that sounds grounded but references nothing outside itself. “Claude confirmed this” is a closed loop. “I’m 85% confident” is too.

Uncertainty Has Structure

Real uncertainty isn’t a single number. It has shape:

• What did I check vs. not check?
• What’s verified vs. assumed?
• What’s stable vs. likely to change?
• What’s primary evidence vs. someone else’s claim?

Collapsing these into one number destroys information. “85% confident” hides whether the agent tested anything, what it assumed, what it didn’t look at.

Structured Uncertainty

Instead of asking “how confident are you,” ask for the structure:

Confidence score:
  "85% confident the auth flow is correct"

Structured uncertainty:
  Tested: login, logout, token refresh
  Not tested: expired token edge case, rate limiting under load
  Assumed: Redis session store matches documented behavior
  Evidence: Ran test suite (14 pass), read auth middleware (auth.go:34-89)

The confidence score tells you nothing actionable. The structured version tells you exactly what to probe if you care.

Why This Matters for Agents

When an agent hands off work, you need to know where to look. “I’m pretty sure this works” gives you nothing. “I didn’t test the webhook retry logic because I couldn’t mock the external service” gives you a specific gap.

When verification fails, you need forensics. “It was 90% confident” explains nothing. “Assumed the config loader handled missing keys, it didn’t” explains everything.

When agents work in sequence, each one inherits the previous one’s assumptions. Structured uncertainty makes the inheritance explicit. Confidence scores bury it.

The Practical Shift

Stop asking agents for confidence levels. Start asking:

• What did you test?
• What did you assume?
• What didn’t you check?
• Where’s the evidence for your claims?

This is harder. It feels like more work than getting a number. But the number is a lie that feels efficient. The structure is truth that feels expensive.

The cost of the lie shows up later, when something breaks and you’re staring at “87% confident” trying to figure out what went wrong.

The Pattern

Confidence scores fail the same test as “tests pass” - they collapse structure into a summary that loses the actionable information.

“Tests pass” doesn’t tell you what’s tested, what’s mocked, what edge cases exist. Green checkmark, no insight.

“85% confident” doesn’t tell you what’s verified, what’s assumed, what’s unknown. Number, no insight.

Both feel like information. Neither helps you act.

Confidence scores are closed loops - they reference nothing outside themselves. Structured uncertainty tells you what was checked, what was assumed, and where the gaps are. One feels precise. The other is useful.