Governance
Agents fail in production for boring reasons
Not hallucination. Expired credentials, a schema the model half-remembered, a tool that silently returned nothing, and a retry that ran the same payment twice.
The public conversation about agent reliability is almost entirely about the model. The incidents are almost entirely about everything else. When an agent that worked for six weeks stops working, the cause is rarely a clever failure of reasoning.
The actual list
- A credential expired, and the tool's error came back as a string the model treated as data - so the agent reported the outcome of an action that never happened.
- The agent produced output in the shape it was asked for on nine turns out of ten, and the tenth broke the consumer downstream.
- A tool returned an empty result and the model read the emptiness as an answer.
- A run was retried after a timeout, and the side effect it had already committed happened twice.
None of these are interesting. All of them are the reason someone turns the agent off.
Failure has to be a row, not an absence
The first thing that goes wrong with agent monitoring is counting only what succeeded. A dashboard built from successful runs looks complete and under-reports failure, because a turn that died is simply missing from it. You cannot see the shape of your failures in a table that only records the wins.
So every settled turn writes a row - its outcome, the tools it called, how many, how grounded the answer was, whether anything had to be repaired mid-turn. The row holds facts, never message bodies, which is what makes it safe to keep long after the transcript has aged out.
Repair beats refusal
When a turn produces output that does not match the shape the agent declared, or prose that cites a document the turn never retrieved, the useful move is usually not to fail the run. It is to tell the model what is wrong and let it fix it once - and then, if the second attempt still misses, to let the answer stand with the problem recorded rather than throwing away work the person is waiting on.
An agent that refuses on every imperfection is an agent nobody leaves running. One that repairs quietly and records the residue is one you can improve.
The recorded residue is the point. Every unrepaired finding is a countable fact against a real denominator, cut by the configuration revision that was live when the turn ran - which is the only way to answer whether last week's change made things better or worse. A flat average across a window that spans a fix averages the fix with the bug it fixed.