I wanted to share this manuscript (https://arxiv.org/pdf/2503.15382). It builds on a rich lineage of literature on the issues with sensitivity and specificity in medical diagnosis.  Unlike this work, we put the focus on the downstream clinical error. We name this clinical error the information mismatch. We also comment on how this mismatch has been exacerbated by the more-readily available pre-test probability calculators.  

This paper was borne out of my own experience trying to compute post-test probabilities when I returned to clinical training. It was also influenced by my experiences trying to talk about this with providers. And, finally, it was maybe based on some frustration I felt with the fact that, despite the many papers criticizing constant test characteristics, most people had never thought about it.  This was meant to rigorously show the issue and translate it into a familiar language (the idea of an information mismatch draws upon a connection to well known medical concept of perfusion-ventilation mismatch).  We have found so far in practice that the idea of a mismatch gives a convenient way to describe the issue, especially from a medical statistics education angle.

Out of concern that the paper might cause people to become exasperated and abandon probability altogether, we also list possible solutions. In these solutions, I personally diverge a bit from those who argue for eradicating sensitivity and specificity altogether, because I do think that although I have some trouble seeing their utility for post-test probability, they may be useful in other ways.

Note also that the information mismatch might also occur if one has sensitivity and specificity that are adjusted surreptitiously. This might occur if, for example, a study that estimates “sensitivity’’ only does so in a subset of the population.  For example, many studies restrict their cohort to adults or patients with severe disease.  Ultimately, we emphasize that regardless of the etiology, an information mismatch is an issue, and the potential for it to lead to a serious clinical error increases along with the availability of pre-test probability calculators.