The AI safety net that nods along while patients are harmed

A real-world study of production AI clinical scribes finds 1 in 20 notes contain errors serious enough to cause significant harm. These are not obvious hallucinations, but subtle mistakes: a headache note omits jaw pain on chewing, which would have revealed giant cell arteritis, a blindness emergency. The note is factually correct, but

dangerously incomplete. The core problem is taste: knowing what matters in a specific context. A dropped line about a patient's holiday is noise in one case and the diagnosis in another. Current evaluation systems fail because they rely on pre-specified rubrics that cannot capture this contextual judgment. The best automated judges pass

one in five seriously flawed notes. Verification is not easier than generation for these problems. Unlike code where unit tests exist, there is no free verifier for safety or completeness. The hard part is not spotting differences between transcript and note, but judging which differences matter. This requires tacit, contextual, and moving

knowledge that cannot be written down in advance. The solution is a continuous evaluation loop: discover failure modes from real outputs, capture expert judgment as examples, and calibrate each output against similar past cases. This approach outperforms static rubrics and fine-tuned models because it learns and adapts to evolving standards without retraining.