The Second Reader Should Show Its Work

The tools I've kept using long-term all share one property: when they flag something, I can see what they saw. A highlighted region, a comparison to a prior study, a texture pattern I can independently evaluate against what I already know. The tools I've abandoned all shared the opposite property — a probability score with no visible reasoning behind it.

It would be easy to assume the second kind is just a worse version of the first, and that better accuracy would fix it. I don't think that's right. Even a highly accurate probability score, offered without reasoning, changes how a radiologist reads. You stop looking as carefully, because some part of your attention has already outsourced the verdict.

That's a real cost, and it doesn't show up in the accuracy benchmarks vendors publish, because those benchmarks measure the tool in isolation, not the radiologist using it. A system can improve stand-alone accuracy while making the human-machine team worse, if the human starts looking less.

What I want from a second reader — human or otherwise — isn't agreement. I want it to point at something and let me decide whether that something matters. That's a genuinely different design goal than maximizing the probability score's correlation with ground truth, and I think it's the one the field should have optimized for from the start.

The best compliment I can give a diagnostic tool is that it's taught me to notice something I'd have missed. That only happens when it shows its work. A probability I either accept or override teaches me nothing — it just asks me to trust it, which is a different request than helping me see.