2020 Volume 43 Issue 10 Pages 1476-1480
Various sources of information are available for identifying and evaluating adverse drug reactions (ADRs). However, some studies only used the ADR data from spontaneous reporting databases to evaluate the safety of post-marketing drugs. This study was performed to identify an appropriate method for evaluating the safety of post-marketing drugs by comparing the frequencies of ADRs among three datasets: randomized controlled trials, published case reports, and spontaneous reports. Taking ADR data for fluorouracil as an example, we collected the three types of data and extracted their ADR information. All listed ADRs were sorted by frequency from high to low, and the top five ADRs were chosen from each dataset. We assigned an index value of 1.0 to the frequency of one specific ADR (diarrhea) and then calculated the index values of the other ADRs relative to diarrhea. Ten different ADRs were mentioned in the top five ADRs of the three datasets, and only diarrhea and nausea/vomiting were included in all three datasets. The rank orders of the top five ADRs varied among the three datasets. Nausea and vomiting was the most frequent ADR in all three datasets; the remaining ADRs differed among the datasets. There were significant differences in the recording of ADRs and the frequency distributions among the three datasets. A comprehensive and reliable safety profile for post-marketing drugs should not be based on any one source. Spontaneous reports from monitoring institutions provided the most ADR data. Randomized controlled trials and case reports published in the literature can supplement the results from spontaneous reports.
Drug safety monitoring is a long and complicated process. Because of the known limitations of clinical trials in detecting drug safety by rare adverse drug events in the pre-market period, safety monitoring of post-marketing drugs is still very important. Consequently, post-marketing safety data are a valuable source of information for early detection of safety signals,1) and the main post-marketing safety data are adverse drug reactions (ADRs). There are various sources of information for identifying and evaluating ADRs. The international community has monitored ADRs since the 1960s to ensure their timely and accurate detection. This monitoring has been mainly based on the spontaneous reporting of ADRs, which remains the main method used by pharmacovigilance centers to generate safety signals.2) Case reports published in the literature can also provide important evidence for drug withdrawal from the market and play an important role in predicting drug safety.3) Furthermore, meta-analyses of randomized controlled trials (RCTs) can be used to analyze multiple studies and are frequently used for the safety evaluation of clinical drug trials.4)
Each of these data sources has its own role in evaluating post-marketing drug safety. When evaluating the safety of post-marketing drugs, some studies5–7) used the ADRs from spontaneous reporting databases such as VigiBase (the WHO database of ADRs) or other spontaneous reporting databases as the data source for their research. However, the ADRs from RCTs and case reports were not within the scope of these studies. To the best of our knowledge, despite the differences among these three datasets, few studies have directly compared ADR data derived from different sources. Additionally, the differences among them in terms of recording ADRs and the frequency distributions of the recorded ADRs remain unclear. We therefore selected three sources of ADR data for one representative drug and compared these three sources to determine if any one source could reliably represent the overall ADR profile for the given drug. We also attempted to establish an appropriate method of obtaining a more comprehensive and reliable safety profile for evaluating the safety of post-marketing drugs.
Malignant tumors are a problem worldwide, and gastric cancer is one of the most common malignancies. In recent years, the use of medications and related drug safety reports have gradually increased in line with the increasing incidence of cancer. According to the Third Edition of the National Comprehensive Cancer Network Clinical Practice Guidelines for Gastric Cancer in 2015, fluorouracil (FU) is one of the drugs of choice for the treatment of gastric cancer. However, patients who take such chemotherapy drugs for a long time are prone to ADRs due to drug accumulation reactions. We therefore analyzed the ADR data for FU as an example because FU is a commonly used drug for the treatment of advanced gastric cancer and sufficient data for this drug are available to support this study. We therefore compared the ADR data for FU derived from RCTs, published case reports, and spontaneous reports from the Chinese National Centre for ADR Monitoring. Our study was specifically designed to compare the frequencies of well-established reactions to FU as a test of the different data sources; we did not intend to research the roles of the three different sources in uncovering new adverse reactions to a freshly launched drug.
ADR data for FU were collected by the Chinese National Center for ADR Monitoring from 2010 to 2014 (we were unable to access earlier data because the Chinese National Center for ADR Monitoring system had been upgraded). The national program for ADR reporting in China was established by the Ministry of Health in 1989. As of 2013, 1 national, 34 provincial, and more than 400 municipal centers for ADR monitoring were included in the 4-level pharmacovigilance network (national, provincial, municipal, and county) with more than 200000 grassroots organization users.8) In China, ADRs are reported to the regional centers and then forwarded to the national center.9)
In the current study, we excluded reports of patients taking FU for reasons other than gastric cancer as well as incomplete and duplicate reports.
Case ReportsCase reports of patients treated with FU for gastric cancer were collected from the above-mentioned sources. We selected articles that included patients aged ≥18 years diagnosed with gastric cancer, including typical case report information and specific ADR manifestations for one or more patients using FU. Case reports were excluded if the patient(s) had serious heart failure, severe liver or kidney dysfunction, or incomplete records of important data. Multiple reports of the same case(s) were also excluded.
RCTsData on RCTs of FU for gastric cancer were collected from the database of Chinese science and technology journals, biomedical literature database, Embase, PubMed, OVID, and the Cochrane Library for the years 1980 to 2015. The search strategy followed the requirements of the Cochrane System Evaluation Manual 5.1.0. Information was restricted to English or Chinese. We selected articles based on the following inclusion criteria: (1) patients aged ≥18 years diagnosed with gastric cancer and included in an RCT, blinded or not; (2) observation group treated with FU monotherapy and control group treated with conventional chemotherapy, with any dose and course of treatment; and (3) detailed indexes of therapeutic evaluation, ADR incidence, or adverse effects. Articles were excluded if they involved patients with serious heart failure, severe liver or kidney dysfunction, or other serious conditions. Studies with no control group and non-RCTs were also excluded.
Statistical AnalysisWe extracted the ADR information from the three types of data and entered them into an Excel worksheet. Non-standard ADR terms in the three datasets were standardized with reference to the WHO Terminology of Adverse Drug Reactions. We sorted the ADRs from high to low frequency and analyzed the five most frequently reported ADRs.
Comparison of Three Data SourcesIt was not possible to determine the incidence of the ADRs in the absence of information on the number of people exposed. Therefore, we compared the relative indexes of ADR frequencies among the three datasets. All listed ADRs were sorted by frequency, and the five most frequent from each dataset were analyzed. Diarrhea was among the top five ADRs in all three datasets, and we therefore assigned it a frequency index value of 1.0. We then calculated the index values for the other ADRs by dividing their frequency by the frequency of diarrhea10) (e.g., if there were 41 reports of hand-foot syndrome and 64 reports of diarrhea among the spontaneous reports, the index value for hand-foot syndrome would be 0.64 (41/64 = 0.64)).
There were 20210 spontaneous reports of ADRs for FU, and this number was reduced to 2951 after data cleaning. The number of cases of the top 5 ADRs was 2172, accounting for 73.60% (n = 2951) of the total frequencies for FU. The index values are shown in Table 1.
| ADR phenomenon | RCTs | Published case reports | Spontaneous reports | |||
|---|---|---|---|---|---|---|
| Total number | Index value | Total number | Index value | Total number | Index value | |
| Diarrhoea | 16 | 1.00 | 26 | 1.00 | 140 | 1.00 |
| Nausea and vomiting | 121 | 7.56 | 52 | 2.00 | 1466 | 10.47 |
| Reduced hemoglobin | 61 | 3.81 | — | — | — | — |
| Oral mucositis | 30 | 1.88 | — | — | — | — |
| Thrombocytopenia | 14 | 0.88 | — | — | — | — |
| Leukopenia | — | — | 6 | 0.23 | 213 | 1.52 |
| Digestive system response | — | — | 10 | 0.38 | — | — |
| Difficulty breathing | — | — | 7 | 0.27 | — | — |
| Bone marrow suppression | — | — | — | — | 269 | 1.92 |
| Phlebitis | — | — | — | — | 84 | 0.60 |
ADRs, adverse drug reactions; RCTs, randomized controlled trials.
According to the selection criteria, we initially generated 43 articles for FU, including 164 cases of ADRs. The number of cases of the top 5 ADRs was 101, accounting for 61.59% (n = 164) of the total frequencies for FU. The index values are shown in Table 1.
RCTsWe initially screened articles according to the recommended literature-screening process in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In total, 741 articles were generated; of these, 11 articles11–21) were finally included for FU. The number of cases of the top 5 ADR was 242, accounting for 64.02% (n = 378) of the total ADR frequencies for FU. The results are shown in Table 1.
The rank orders of the ADR index values from the three datasets are shown in Fig. 1.
ADRs, adverse drug reactions; RCTs, randomized controlled trials.
The present study showed that the types of the top five ADRs varied among the three datasets (Table 1). In total, 10 different ADRs were mentioned in the top 5 ADRs of 3 datasets, but only diarrhea and nausea/vomiting were included in the top 5 ADRs of these 3 datasets. This result indicates that there were differences in the main types of recording ADR that from above three datasets.
However, regarding the frequency distribution, the rank orders of the top five ADRs varied among the three datasets (Fig. 1). Nausea and vomiting was the most frequent ADR in all three datasets, and the other ADRs were different. However, the relative values for the same ADR were also inconsistent: the index values of nausea and vomiting were 7.56, 2.00, and 10.47 according to the RCTs, case reports, and spontaneous reports, respectively. Furthermore, in the spontaneous reports, nausea and vomiting was 10.47 times as frequent as diarrhea, and it was 7.56 and 2.00 times as frequent in the RCTs and case reports. These results highlight the differences among the three datasets in terms of recording ADRs and the frequency distributions by these intuitive numbers. The ADR information from spontaneous reports does not contain ADR information provided by RCTs and case reports.
This conclusion is similar to that reported by Loke et al.10) The rank orders of all ADRs differed among the three datasets from two studies. In terms of frequency, however, the two studies were still slightly different. In the present study of FU, nausea and vomiting was the most frequent ADR in all three datasets. In the study of amiodarone by Loke et al., however, the most frequent ADR in the three datasets was completely different: thyroid disorders were the most commonly reported ADRs in the spontaneous reports. In contrast, published case reports showed a preponderance of respiratory disorders, while cardiac conduction problems were the most frequent ADRs in the RCTs.
These differences may be caused by several factors. First, the differences in the results may be due to the essential difference between amiodarone and FU. Amiodarone is a broad-spectrum antiarrhythmic drug, and FU is used for gastric cancer. Therefore, the ADRs of the two drugs differ in terms of type and frequency. Second, with the rapid development of data storage capabilities and statistics,22) increasingly more technologies such as the Internet and big data are being used to collect ADR information. Since the study of amiodarone by Loke et al. in 2003, the number of ADRs collected from the three data sources has been gradually increasing, and the quality is constantly improving. The gradual improvement in the quality of information obtained from the three ADR databases may thereby result in some commonality among the three sources; that is, the most common ADR among the three was consistent in the present study. However, other aspects remain inconsistent.
According to the total number of ADRs recorded in these three datasets, there were 2951 spontaneous reports of ADRs, compared with 378 and 164 reports from RCTs and case reports, respectively. Spontaneous reports generally provided more reports of ADRs than either of the other two sources. The spontaneous reporting system is an important ADR monitoring tool worldwide. Spontaneous reports reflect real-world data, and they can also detect rare adverse events not found in clinical trials or other pre-marketing studies. Data-mining techniques are being increasingly applied to traditional reviews of these reports to allow rapid analysis of large volumes of accumulated data.23,24) Spontaneous reports can play an important role in identifying post-marketing drug safety problems. However, the ADRs from RCTs and published case reports are limited in number and scope. Consequently, spontaneous reports are currently the most important source of data for evaluation of ADRs. Hence, some studies have used the ADRs from spontaneous reporting databases as their data sources when evaluated the safety of post-marketing drugs.
Spontaneous report-based studies also have several limitations. Under-reporting (failure to report a suspected ADR) and selective reporting are considered the main limitations of a pharmacovigilance system.25,26) Moreover, estimating the risk of a given ADR requires adequate information on drug utilization, and Motola et al.25) showed that information from sales data may not accurately reflect usage and prescribing levels.
However, because of the huge amount of data obtained from spontaneous reports, it is impossible for researchers to analyze all types of ADRs one by one, and the most frequent or severe ADRs are therefore usually chosen for key analysis. For example, the top five ADRs from spontaneous reports in the present study were nausea and vomiting (10.47), bone marrow suppression (1.92), leukopenia (1.52), diarrhea (1.00), and phlebitis (0.60). These ADRs will inevitably be analyzed by most researchers. In contrast, ADRs such as reduced hemoglobin and oral mucositis may be neglected because they account for a small proportion in the spontaneous reports. However, the index values of reduced hemoglobin and oral mucositis were 3.81 and 1.88, respectively, in RCTs, and their frequencies were ranked second and third, respectively. When using RCT data to analyze the ADRs of FU, reduced hemoglobin and oral mucositis will become the focus. Consequently, the two conclusions drawn by the information obtained from spontaneous reports and RCTs are not consistent.
RCTs generally only assess drugs in small samples during a relatively short follow-up period; therefore, the capacity of RCTs to detect safety problems is limited.27) However, based on their rigorous experimental design, RCTs are regarded as the gold standard for assessing the effectiveness of medications. ADR reports based on RCTs are thus of high quality. Additionally, meta-analyses can be used to retrospectively analyze the effects of multiple studies; synthesize new knowledge based on existing RCT data; and provide a systematic, objective, and comprehensive approach to a problem. Thus, meta-analyses may help to provide more reliable estimates of ADRs.28)
Published individual case reports provide detailed descriptions of ADRs and can thus offer vital clues for identifying rare but serious ADRs. Such reports may also generate hypotheses that can direct ongoing scientific research and suggest future research directions for the study of ADRs and phytotoxicity events.29) Furthermore, case reports of ADRs are likely to report unique, unusual, or severe features, which have great research value. In addition to routine reporting of case reports, their potential additional value should also be explored.30) For example, one of the ADRs mentioned in this study—respiratory depression, which ranked fifth in the case reports—had an index value of 0.27. Although its incidence is low, it has a great impact on patients. Regardless of what kind of data are found and what type of database such data are obtained from, researchers will focus on this kind of information. However, case reports can provide more detailed information and clues based on the other two databases.
Therefore, although RCTs and published case reports are limited in number and scope, they still provide indispensable evidence for evaluating ADRs. A comprehensive and reliable safety profile for post-marketing drugs should not be based on any one source. Spontaneous reports from monitoring institutions provide the most ADR data, and this dataset is the most important source of data for evaluation of ADRs. Data-mining techniques can be used to analyze these accumulated data and thus evaluate the safety of post-marketing drugs, such as when producing a drug safety-warning model to support the rational use of drugs. In addition, for ADRs extracted from RCTs, a meta-analysis can be used to obtain a quantitative result, and the ADRs extracted from case reports can be used to analyze the details of the occurrence of ADRs.
This study had certain strengths and limitations. Its main strength was the direct comparison of the three different data sources based on the index values of ADR frequencies. A limitation of this study is that we used RCT and case report data from 1980 to 2015 but only used spontaneous reports from 2010 to 2014. This occurred because the Chinese National Center for ADR Monitoring system had been upgraded and we were therefore unable to access earlier data. Additionally, the literature search was limited to Chinese and English. Because of the limitation of data sources, the spontaneous reports only included data from China; spontaneous reports could not be obtained from worldwide sources, which may have impacted the research results. Further studies with more data would be useful to confirm the current findings.
There are significant differences in terms of recording ADRs and the frequency distributions among the three datasets evaluated in this study. The ADR information from spontaneous reports does not contain ADR information provided by RCTs and case reports. A comprehensive and reliable safety profile for post-marketing drugs should not be based on any one source. When evaluating the safety of post-marketing drugs, spontaneous reports from monitoring institutions can provide the most ADR data. In addition, ADR reports based on RCTs are of high quality, and the type and proportion of ADRs are different from those of spontaneous reports; thus, such ADR reports should also be included. Finally, the ADRs extracted from case reports can be used to analyze the details of the occurrence of ADRs. In conclusion, RCTs and case reports published in the literature can supplement the results from spontaneous reports when evaluating ADRs.
This work was supported by the Project of the National Social Science Foundation of China (No. 13BGL132) and the Social Development Science and Technology Research Foundation of Shaanxi Province (No. 2016SF-159).
The authors declare no conflict of interest.
