Vaccine Effectiveness Without Matching: Why Calendar Time Comes Before Pairing
Anas H. Alzahrani, MD PhD MPH
Department of Preventive Medicine and Public Health
Faculty of Medicine, King Abdulaziz University
Vaccine effectiveness without matching can be estimated credibly—but only if the causal question comes before the machinery. Calendar time is not a formatting variable. It determines who could be vaccinated, who remained unvaccinated, which pathogen was circulating, and how much infection pressure each person faced.
That creates a moving comparison. A person vaccinated during a wave and an unvaccinated person observed months earlier may look like two treatment strategies, but they also experienced two epidemics. Adjustment cannot rescue a study until the design says which calendar-time contrast it intends to estimate.
Why the Calendar Belongs in the Causal Question
A target trial needs eligible participants, treatment strategies, time zero, follow-up, an outcome, a causal contrast, and an analysis plan. In a vaccine study, calendar time touches nearly every element. Eligibility may expand by age or risk group. Uptake accelerates after policy changes. Infection hazards rise and fall. Variants, prior immunity, testing availability, and background prevention can change too.
Entry
Who is eligible and willing to vaccinate changes across rollout phases.
Exposure pressure
The same follow-up duration can carry very different infection risk in different weeks.
Meaning
A pooled effect averages over some calendar distribution, whether or not the paper names it.
The key question is not merely whether calendar date appears in a regression model. It is whether the vaccinated and comparator strategies begin under comparable conditions and whether the final estimate averages effects over a calendar distribution that readers can understand.
Interactive calendar-time explorer
Watch One Vaccine Effect Become Two Estimates
Assume the vaccine has the same relative effect in quiet and peak periods. Unvaccinated comparator observations are 30% peak-period and 70% quiet-period. Change the epidemic peak, the within-period vaccine effectiveness, and when vaccinated observations enter the study.
Crude pooled comparison
19.4% VE
Vaccinated risk 5.0% versus comparator risk 6.2%, with each group carrying its own calendar-time mix.
Same calendar-time distribution
60.0% VE
Both strategies are standardized to the same 50:50 mix of quiet and peak periods, recovering the selected within-period effect in this simplified example.
Calendar mixing materially changes the headline
The crude estimate and the calendar-standardized estimate answer differently weighted questions. The study must say which one it intends.
Teaching model only. This deterministic illustration assumes no confounding within a calendar period, one binary outcome, constant relative vaccine effectiveness, and no sampling uncertainty. Real analyses must also address eligibility, treatment timing, censoring, prior infection, outcome ascertainment, and effect modification.
The Two Clocks a Vaccine Study Must Align
Vaccine-effectiveness analyses often carry two time scales. Time since vaccinationdescribes biological follow-up: day 7, day 30, month 6. Calendar time describes the epidemic and rollout: January 15, the first week of a wave, or the period when a new variant dominated. They answer different questions and are usually correlated.
| Clock | What it carries | Failure if ignored |
|---|---|---|
| Time since vaccination | Onset and waning of protection | Early immune-response time or later waning is averaged away |
| Calendar time | Incidence, variants, policy, testing, background immunity | Different epidemic conditions are attributed to vaccination |
| Age or disease time | Changing susceptibility and clinical history | Underlying risk is treated as fixed during follow-up |
Choosing one clock as the analysis time scale does not make the others disappear. A sound design explains how they enter eligibility, risk-set construction, modeling, standardization, and interpretation.
Clinical Example: Rollout Meets a Winter Wave
Imagine a pediatric vaccine becomes available in November. Early uptake is modest while infection pressure is low. By January, uptake has accelerated—but so has community transmission. A crude cohort compares many vaccinated child-weeks from January with many unvaccinated child-weeks from November.
Even if vaccination lowers risk within every week, the vaccinated group may show a higher pooled infection risk because more of its follow-up occurs during the wave. The reverse can happen if rollout coincides with a falling epidemic. Neither distortion requires a flawed laboratory test or a dramatic patient-level confounder. The calendar itself changed the background hazard and the composition of the risk sets.
The clean metaphor
Comparing strategies across unmatched dates is like comparing two umbrellas in different storms. The outcome reflects the umbrella and the weather. A causal design has to put both under the same sky.
Matching Is a Tool, Not the Estimand
Matching vaccinated people to unvaccinated comparators on a nearby date can make the design easier to explain, but matching does not define the causal quantity by itself. Choices about replacement, match ratios, calipers, re-use of future vaccinees, and censoring at later vaccination determine which people and dates receive weight. Discarded unmatched participants can also change the population represented.
A 2026 methods paper by Wu and colleagues makes this distinction explicit. The authors define a vaccine- effectiveness estimand that summarizes effects over calendar time, identify it under stated assumptions, and estimate it with hazard-regression models rather than matching. In simulations and a pediatric COVID-19 vaccine application, their estimators produced similar scientific conclusions with greater precision than the matching approaches they evaluated.
The practical lesson is not that matching is always wrong or that hazard models are automatic winners. It is that the target estimand should be written before choosing the machinery. Reviewers need to know what calendar dates, eligible populations, and treatment contrasts the reported average represents.
Four Design Decisions That Make the Estimate Legible
- Define repeated eligibility. State when each person can enter an emulated trial and whether the same person may contribute to later trials.
- Align time zero. Compare vaccination and no vaccination among people eligible on the same date, with treatment assignment and follow-up beginning together.
- Name the calendar-time target. Say whether effects are averaged over the observed rollout, a fixed policy-relevant period, or another prespecified distribution.
- Report absolute risk by recognizable periods. A stable relative effect can coexist with dramatically different numbers of infections prevented when background incidence changes.
These decisions should appear in the protocol table, not only in model syntax. If the study emulates a sequence of weekly trials, readers should see the weekly eligibility rule, treatment strategies, follow-up, censoring, and how those trials are combined.
A Reviewer Checklist for Calendar-Time Claims
- What is time zero for each eligible person, and what calendar dates contribute to the comparison?
- Could a person contribute as unvaccinated before vaccination and later become eligible for a vaccinated strategy?
- How did infection pressure, circulating variants, testing, and public-health policy change during follow-up?
- Are vaccinated and comparator observations drawn from the same calendar-time risk sets?
- What calendar-time distribution does the reported vaccine-effectiveness estimate average over?
- Were absolute risks reported for clinically recognizable dates or periods, not only one pooled relative effect?
- Do sensitivity analyses change the time scale, period definition, or model for calendar-time trends?
If the answers are buried, start with the person-time table. Count vaccinated and comparator follow-up by week or month alongside outcome incidence. A large imbalance does not prove bias, but it reveals the calendar-time modeling and standardization assumptions that the paper must defend.
Common Reassurances That Do Not Settle the Design
- “We adjusted for calendar month.” A coarse category may not capture a sharp wave, and model adjustment still does not explain the target calendar distribution.
- “The matched dates were close.” Proximity helps only if risk and measurement are sufficiently stable inside the matching window and the resulting matched population answers the intended question.
- “The hazard ratio was adjusted.” Adjustment variables do not reveal who was eligible, when strategies began, or how an effect was averaged across dates.
- “Effectiveness was consistent across periods.” Wide intervals, changing outcome definitions, or sparse risk sets can make apparent consistency uninformative.
Calendar time may act as a confounder, an effect modifier, a standardization variable, or all three. The role follows from the question and data-generating process; it cannot be settled by checking a covariate box.
Where Aqrab Fits
Vaccine-effectiveness papers can look technically mature while leaving their causal calendar implicit. Aqrab helps reviewers translate the methods into a target-trial question: who was eligible on each date, which strategies were compared, whether time zero aligned, and what population-and-time average the reported estimate can support.
If you are reviewing an observational vaccine study, try Aqrab on the design and analysis sections. For structured critique inside a research workflow, see the developer documentation.
Sources and Further Reading
- Wu E, Rogawski McQuade E, Stensrud MJ, Nabi R, Benkeser D. Target Trial Emulation Without Matching, on defining and estimating vaccine effectiveness over calendar time.
- Hernán MA, Robins JM. Using Big Data to Emulate a Target Trial, on specifying the trial protocol before analyzing observational data.
- Aqrab guide to target trial emulation, for the core protocol components.
- Aqrab guide to test-negative vaccine-effectiveness designs, for selection through testing pathways.
The Practical Bottom Line
Vaccine uptake and infection pressure share a clock. If a study compares people observed under different epidemic conditions, one pooled estimate can mix the intervention with the weather around it.
Put calendar time in the target trial, align eligible risk sets, state how effects are averaged, and show absolute risks across recognizable periods. The calendar is not housekeeping. It is part of the causal question.
Keep reading
Don't stop at one method.
Good methods judgment comes from contrast. Read the neighboring guides, see where the assumptions diverge, and avoid treating every observational problem like it needs the same hammer.
Target Trial Emulation Cannot Randomize Clinical Judgment: A Pertussis Study Audit
A practical target-trial emulation audit using an infant pertussis study. Check clinical-judgment confounding, propensity-score overlap, endpoint timing, sparse outcomes, and claim strength.
Comparator Selection in Observational Studies: Why the Control Group Changes the Question
A practical guide to comparator selection in observational studies. Learn how active comparators change the estimand, confounding structure, and interpretation of real-world evidence.
Treatment-Timing Effects: When “Earlier Is Better” Needs a Fair Clock
A practical guide to auditing earlier-is-better claims in observational studies. Test time zero, treatment strategies, evolving clinical decisions, positivity, and timing curves before reading a treatment gradient as biological.