Validity of vaccination information in the COVID-19 surveillance system in Japan: Implications for developing efficient and highly valid data collection systems in future pandemics

Abstract

Japan's government developed the Health Center Real-time Information-sharing System on COVID-19 (HER-SYS), the national COVID-19 surveillance system, which relies on manual data entry. Following the COVID-19 vaccination campaign, physicians were mandated to report COVID-19 cases with vaccination history via HER-SYS. However, concerns have arisen regarding the accuracy of this vaccination history. This study aimed to assess the validity of vaccination history recorded in HER-SYS. We used data from HER-SYS provided by three municipalities. The study cohort comprised COVID-19 cases registered in HER-SYS from February 2021 to March 2022. The validity of vaccination history in HER-SYS was assessed by cross-referencing with the Vaccination Record System (VRS) of these municipalities. We calculated sensitivity to gauge the extent of missing data in HER-SYS, and positive predictive value (PPV) to evaluate the accuracy of data entered into HER-SYS. Of the 19,260 COVID-19 cases included in the study cohort, HER-SYS and VRS identified 3,257 and 8,323 cases, respectively, as having the first-dose vaccination history. Cross-referencing identified 3,093 cases as true positives in HER-SYS. The sensitivity was 37.2% (95% confidence interval [CI]: 36.1−38.2) and the PPV was 95.0% (95% CI: 94.2−95.7). Collection of vaccination data by HER-SYS was found to be inadequate to obtain information on vaccination history of COVID-19 cases. This suggests that real-time data linkage across different systems such as HER-SYS and VRS would reduce the burden of manual data entry during the pandemic and lead to appropriate infection control measures based on more accurate information.