Who Knew the Answer? Reader Masking in Diagnostic Accuracy Studies
October 22, 2026 · 5 min read · Aqrab editorial team
A scan is read as suspicious. Later, a pathologist classifies the tissue. A paper compares the two results and reports diagnostic accuracy. Before accepting its numbers, ask a deceptively simple question: what did each reader know when making the call? The order of testing alone is not enough if images or specimens were interpreted after other results became available.
Two directions of information flow
The index test is the test being evaluated; the reference standard establishes target-condition status as well as the study can. Information can flow in either direction.
| Who interprets? | Information that may leak | Question for the report |
|---|---|---|
| Index-test reader | Reference-standard result | Was the test interpreted and locked before the answer key was known? |
| Reference-standard assessor | Index-test result | Could the result under evaluation influence the final condition label? |
Either direction can make apparent agreement less independent, especially when a judgement has room for interpretation. But an unmasked design does not prove that any particular accuracy estimate is biased by a known amount. The risk depends on what information was available, when, and how the result was determined.
The practical takeaway
For both sides of a diagnostic comparison, record reader, timing, information available, and when the interpretation was locked. “Test performed first” is not the same as “test read before the reference result.”
Clinical context is a separate question
A radiologist may normally see age, symptoms, or prior imaging. Removing all of that context can change the task being studied. The relevant question is not “Was every fact hidden?” but “Did readers receive the information they would have in intended use, and were they shielded from the other test's answer?” Report those two facts separately. An artificially context-free reading may have limited applicability even when it reduces one route of bias.
A four-line methods audit
- Index reader: Identify who made the call and whether they knew reference-standard findings.
- Reference assessor: Identify who assigned condition status and whether they knew the index result.
- Timing: Distinguish acquisition from interpretation; say when each result was finalized.
- Usual care: List ordinary clinical information available to readers and whether it matched intended use.
If a paper says only “blinded,” ask who was blinded to what. If it says nothing, treat the information flow as unknown, not as proof of either masking or leakage.
Where Aqrab fits
Use Aqrab to pressure-test whether a diagnostic paper reports both directions of reader masking, interpretation timing, and intended clinical context. A methods critique can flag a missing answer; it cannot reconstruct who knew what from an unspecified protocol.
Source anchors
STARD 2015 explanation and elaboration asks whether clinical information and reference-standard findings were available to index-test readers, and whether index-test findings were available to reference-standard assessors (items 13a–13b). The original QUADAS-2 paper treats interpretation without knowledge of the other result as a risk-of-bias question in both test domains. Neither guideline supplies a numerical correction for a study with unreported information flow.
Keep reading
Don't stop at one method.
Good methods judgment comes from contrast. Read the neighboring guides, see where the assumptions diverge, and avoid treating every observational problem like it needs the same hammer.
Only the Disagreements Were Checked: The Discrepant-Resolution Trap
When a third test checks only disagreements, agreement is not proof that both tests were right. Use this three-question audit before accepting a revised diagnostic-accuracy table.
The Result Arrived After the Decision: Audit Diagnostic Turnaround
A practical timeline shows why diagnostic accuracy alone cannot establish that a result can reach the clinical decision it is meant to support.
One Pooled Sensitivity, Two Hospitals: Check Diagnostic Accuracy by Site
A worked two-site example shows how a pooled sensitivity can hide a weaker result in one setting, and what to inspect before transporting a diagnostic claim.