Only the Disagreements Were Checked: The Discrepant-Resolution Trap
November 3, 2026 · 5 min read · Aqrab editorial team
A diagnostic paper compares a new test with an older test. When they disagree, the researchers run a third, more authoritative test to decide which was right. The revised table looks cleaner. But what happened when the first two tests agreed? If those cases were never checked, both tests could still have been wrong together. This is discrepant resolution: resolving only the disagreements does not establish accuracy for the whole study population.
A Simple Audit of the Missing Cells
Imagine 100 people receive both the new test and a comparator. The numbers are hypothetical and illustrate a design, not any real test. A reference-standard test is then used only for the 20 people whose results differ.
| New test and comparator | People | Reference-standard result known? |
|---|---|---|
| Agree | 80 | No |
| Disagree | 20 | Yes |
The third test can identify which original result agrees with it among the 20 discordant cases. It cannot tell us how many of the 80 concordant cases were jointly wrong. Re-labeling the entire cohort as though its condition status were established silently assumes that agreement means truth.
The practical takeaway
Before accepting a “resolved” sensitivity or specificity, ask: Were concordant cases also verified by a valid reference standard? If not, do not treat the revised table as a complete accuracy table.
Three Questions for the Paper
- What was the comparator? If it was not a valid reference standard, initial agreement measures are not sensitivity and specificity.
- Who received the resolver? Request the original paired-test table and the number verified in every cell, including agreements.
- How was the final estimate calculated? A revised table that simply replaces discordant comparator results and assumes concordant results are correct is not a valid accuracy estimate.
When full reference-standard testing is feasible, applying it across the study population avoids this particular blind spot. When it is not, a planned probability sample of concordant cases plus appropriate adjusted analysis may support an estimate; naively applying the usual two-by-two formulas to a selectively verified sample does not. The design and analysis need to be reported, not inferred from a polished headline.
Where Aqrab Fits
Use Aqrab to structure a methods critique around the comparator, verification pattern, and denominator behind a diagnostic claim. A critique can surface an unsupported accuracy estimate; it cannot recover unmeasured reference-standard results.
Source Anchors
The FDA statistical guidance for diagnostic-test studies explains why discrepant resolution cannot turn a comparison with a non-reference standard into a valid sensitivity/specificity table, and discusses planned partial verification with adjusted estimation. The STARD 2015 checklist calls for describing the reference standard, participant flow, missing test results, and the methods used to estimate accuracy. Neither source supplies the hypothetical counts above.
Keep reading
Don't stop at one method.
Good methods judgment comes from contrast. Read the neighboring guides, see where the assumptions diverge, and avoid treating every observational problem like it needs the same hammer.
The Result Arrived After the Decision: Audit Diagnostic Turnaround
A practical timeline shows why diagnostic accuracy alone cannot establish that a result can reach the clinical decision it is meant to support.
One Pooled Sensitivity, Two Hospitals: Check Diagnostic Accuracy by Site
A worked two-site example shows how a pooled sensitivity can hide a weaker result in one setting, and what to inspect before transporting a diagnostic claim.
Before the Analyzer: Does Specimen Handling Match Diagnostic Test Use?
A practical specimen-path audit for diagnostic studies: collection, transport, storage, and exclusions can separate the reported accuracy from the intended clinical workflow.