A model that answers badly does not crash
Self-hosted inference — written 27 August 2026, republished here 2 September 2026
Note on republication. Written and published elsewhere on 27 August 2026, before this site existed. It appears as written with one exception: the original described the system as similarity search over an index. That was inaccurate. It is a deterministic extraction pipeline followed by local classification, and the distinction is developed in later notes.
Errors: zero. Latency: 98 ms. Alerts: none.
And the answers are wrong.
A language model that answers badly does not crash. It answers. This cost me more time than anything else in six months of self-hosting, because every conventional monitoring signal reports that all is well.
What works for me is not elegant.
A test set versioned like code
Named owner, replayed on every change of model, prompt, or index. Without that, you cannot tell whether a shift came from the system or from chance.
Human review on a random sample
Random, and not on flagged cases. Review only what you already suspected and you measure your own suspicions, mistaking that for verification.
Disagreement rate treated as a metric in its own right
The gap between the automated judge and the human one. That gap decides when review can be relaxed — not a feeling that things seem fine.
What this actually resembles
Industrial quality control far more than application monitoring. You sample, you compare against a reference, you accept a share of uncertainty and you put a number on it.
The human cost of evaluation does not go away. The organizations that hold up are the ones that accepted this.