SHORT REPORT
Assessment of the Publication Trends of COVID-19 Systematic Reviews and Randomized Controlled Trials
2021 Volume 3 Issue 2 Pages 56-58
Details
2021 Volume 3 Issue 2 Pages 56-58
Randomized controlled trials (RCTs) and systematic reviews (SRs) are the most essential and useful study designs for evidence-based medicine and practice. The publications of SRs and meta-analyses have surprisingly increased, and the number of SRs is larger than that of RCTs in some medical fields [1]. During the COVID-19 pandemic, the number of studies listed in The National Library of Medicine’s registry (ClinicalTrials.gov) and preprints in medRxiv for COVID-19 has grown rapidly [2]. Additionally, many protocols of SRs for COVID-19 are found in PROSPERO, an international prospective register of SRs [3]. However, several reports have expressed concerns regarding unguarded, irrelevant, and misleading SRs and meta-analyses [1, 4]. Therefore, in this study, we have clarified the publication trends of RCTs and SRs regarding COVID-19.
We conducted a cross-sectional study by investigating the number of SRs and RCTs on topics related to COVID-19 practice published in PubMed and medRxiv between January 1, 2020 and October 31, 2020. We excluded study protocols. The search strategies used in PubMed, which included Shokraneh’s filter for COVID-19 [5], are shown in Supplementary Table 1. We retrieved abstracts from medRxiv using the following search terms: “random” for the RCTs, and “review,” “evidence synthesis,” “meta-analysis,” and “metaanalysis” for the SRs. Two of the four review authors (ST, YK, TA, and SO) independently selected abstracts from the filtered search results. Disagreements between authors were resolved through discussion. If necessary, a third review author acted as an arbitrator. We calculated the ratio of the number of RCTs to that of SRs for this study period, as in a previous study [1]. A ratio greater than 1 indicated that more SRs than RCTs were published, whereas a ratio less than 1 indicated the opposite. We made a list detailing the country of affiliation of the included articles’ first author. Ethical approval and formal consent were not required for this study.
Between January and October 2020, 807 SR articles and 49 RCT articles (Supplementary Table 2) on COVID-19 were published in PubMed, while 451 SR articles and 54 RCT articles (Supplementary Table 3) were published in medRxiv. The SR/RCT ratio in PubMed increased from 9.5 in April to 40.0 in September. In medRxiv, the SR/RCT ratio rose from 7.7 in March to 28.3 in May (Figs. 1 and 2, respectively). Many SRs and RCTs published in PubMed and medRxiv were from China (187 SRs and 35 RCTs) and the United States (US [192 SRs and 21 RCTs]) (Supplementary Tables 4 and 5, respectively). A peak in SR and RCT articles from China was followed by a peak in SR and RCT articles in the US (Supplementary Figs. 1–4).
This figure shows the trend in the number of publications of COVID-19-related systematic reviews (including systematic reviews and meta-analyses) and randomized controlled trials published in PubMed. It presents the ratio of COVID-19-related systematic reviews to randomized controlled trials.
Note: SR = systematic review; RCT = randomized controlled trial; N/A = not available
The figure shows the trend in the number of COVID-19-related publications of systematic reviews (including systematic reviews and meta-analyses) and randomized controlled trials in medRxiv. It presents the ratio of COVID-19-related systematic reviews to randomized controlled trials.
Note: SR = systematic review; RCT = randomized controlled trial; N/A = not available
This is the first report to research the publication trends of COVID-19-related SRs and RCTs. The monthly SR/RCT ratios increased by 40.0 and 28.3 in PubMed and medRxiv, respectively. The increases in SR/RCT ratios were much larger and faster than that shown in a previous review of all medical research, which reported the ratio to be 0.871 in 2017 [1]. However, except for Cochrane Reviews, the methodological quality of COVID-19 SRs are generally low [6]. Therefore, the results show that presumably, unguarded, irrelevant, and misleading SRs have been excessively published. Similar to the first peak in the number of COVID-19 patients in each country, there was a peak in SR and RCT articles from China followed by a peak in SR and RCT articles from the US. The present review only considered articles related to COVID-19, which is an emerging infectious disease; this could have been the reason for the aforementioned discrepancy in the SR/RCT ratio of publications, as previous reviews have focused on well-known diseases. A similar proportion of SR and RCT articles was published, and the number of COVID-19 related SRs published in medRxiv increased rapidly, followed by a delayed increase in PubMed. This delay between medRxiv and PubMed can be explained by the time needed for the peer review process. A limitation of this study was that when we counted the number of SRs, we included non-interventional SRs, as in a previous study [1].
In conclusion, the trend in COVID-19-related publications was marked by a rapid increase in the number of SRs compared with RCTs during the study period. Although summarizing evidence is useful for viewing the body of evidence and for making informed decisions about health and social care, the number of unguarded, irrelevant, and misleading SRs and meta-analyses are possibly increasing. Therefore, further research is needed to clarify the yearly changes of COVID-19-related SRs and RCTs.
None
The authors would like to thank Editage (http://www.editage.jp) for their assistance in the English language editing process.
