The Expert-Reader Trap: Will Diagnostic Accuracy Travel to Routine Care?
October 25, 2026 · Aqrab editorial team
A diagnostic paper reports impressive sensitivity and specificity. Its images were read by subspecialists after dedicated training, with time to review each case. The proposed service, however, will use generalists during routine shifts. The paper may have measured its test well in the study setting; it has not yet shown that the same performance holds in the proposed workflow.
The Reader Is Part of the Test
For interpretation-dependent tests, the index test is not just an image, slide, or model output. It includes how the result is produced and interpreted. The STARD 2015 explanation asks authors to report the number, training, and expertise of people executing and reading tests because those details can affect whether results are achievable elsewhere. It also asks for enough procedural detail to assess applicability.
This is an applicability question, not a claim that experts always outperform other readers. Training, clinical information, case volume, reading time, and escalation rules may differ between the study and routine care. Their effects must be measured or explicitly left uncertain, not assumed from a headline accuracy estimate.
The practical takeaway
Before transporting a diagnostic accuracy estimate, write down who interpreted the test in the study, who will interpret it in practice, and which parts of their workflow differ.
A Concrete Comparison
Imagine a hypothetical study in which three specialist readers each complete a structured training set, then independently classify research images in a quiet session. A clinic plans to use the same image type, but a rotating team will interpret it alongside other duties, with no equivalent training set. Even if the patient mix and reference standard were identical, the reader protocol would not be. The published sensitivity and specificity describe the studied combination of test, readers, and conditions; they are not automatic estimates for the clinic.
The direction of any difference is unknown from this scenario. The point is to identify the untested assumption and design an evaluation under intended-use conditions.
Five Questions for Reviewers
- Intended use: Is the test for screening, triage, add-on assessment, or replacement, and who is expected to act on it?
- Readers: How many interpreted the index test? What were their professions, experience, selection criteria, and training?
- Workflow: What information, time, software, assistance, repeat reads, and adjudication were available? Do those conditions match routine use?
- Variation: Are results shown by reader or relevant reader group, with uncertainty, rather than only a pooled estimate? Were comparisons planned before seeing the data?
- Next evaluation: If the intended users were not studied, what prospective or local assessment would test performance in their actual pathway?
A study can be internally sound and still leave this transport question open. Reporting the gap is more useful than treating a well-controlled reading room as a universal deployment setting.
Where Aqrab Fits
Use Aqrab to structure a methods critique around intended use, reader selection, training, and workflow. A critique can flag missing evidence and make a local validation question explicit; it cannot supply performance data for readers who were never studied.
Source Anchors
The STARD 2015 checklist covers intended use, test execution, variability in accuracy, and generalisability. Its explanation and elaboration discusses reader training and expertise as applicability details. The FDA diagnostic-test reporting guidance asks for the intended-use and study populations when interpreting performance measures. The scenario above is hypothetical, not evidence that any particular reader group performs better or worse.
Keep reading
Don't stop at one method.
Good methods judgment comes from contrast. Read the neighboring guides, see where the assumptions diverge, and avoid treating every observational problem like it needs the same hammer.
Two Positive Tests Are Not Always Twice the Evidence
A worked example shows why repeating a correlated diagnostic test cannot automatically multiply the same likelihood ratio twice—and what joint pathway data reviewers should request.
Inconclusive Diagnostic Results: Keep the Middle Row
A three-result worked example shows why valid intermediate findings deserve their own likelihood ratio, while failed tests need separate reporting—not silent exclusion.
Positive Predictive Value: Why a Good Test Can Still Produce Mostly False Alarms
A worked diagnostic-test example shows why positive predictive value changes with the tested population, even when sensitivity and specificity are held fixed—and why real-world accuracy may shift too.