Regular Articles
The Quality of Spontaneous Adverse Drug Reaction Reports in China: A Descriptive Study
2019 Volume 42 Issue 12 Pages 2083-2088
Details
2019 Volume 42 Issue 12 Pages 2083-2088
Pharmacovigilance is important to monitor the safety of drugs. There are, however, problems with the quality of adverse drug reaction reports in China. This study aimed to analyze the quality of adverse drug reaction reports in China, identify the factors affecting it, and propose measures to improve it. In our study, the western province of Shaanxi, the central province of Hubei and the eastern province of Jiangsu were chosen as typical, and adverse drug reaction reports from these three provinces from 2015 to 2017 were systematically sampled. The sampling reports were scored and graded to assess their quality. The results showed that only 10.18% were considered high quality in a total of 3429 reports. There were statistically significant differences in quality by year, province, report type, report source, and occupation of the reporter (p < 0.001). Reports from Shaanxi were slightly poorer quality, and “new” and “serious” reports and those from pharmacists were higher quality. Five indicators were particularly poor quality: patient information, adverse drug reaction, reporter information, drug information and vigilance. In conclusion, the quality of adverse drug reaction reports in China still needs improvement. Factors affecting quality included timing, location, report type, report source, and reporter’s occupation. It may be helpful to publicize the importance of monitoring adverse drug reactions and improve the knowledge of reporters.
Adverse drug reactions (ADRs) affect patients’ physical recovery from illnesses, and can also result in economic burdens. Some recent studies have suggested that adverse drug reactions were the main cause of hospitalization, and can account for up to 5% of hospital admissions.1,2) It is therefore clear that monitoring adverse drug reactions of patients through pharmacovigilance is very important in ensuring drug safety. Collecting ADR information is an easy, low-cost and extensive method to monitor the safe use of drugs.
Comparing with some developed countries, China’s pharmacovigilance system was established relatively recently, with monitoring piloted in 1988 and the China National Center for Adverse Drug Reaction Monitoring (CNADRM) formed in 1989. After the Proposed Regulation on Administrative Measures on Reporting and Monitoring of ADRs and the amendment of the Drug Administration Law of the People’s Republic of China (PRC) in 2001, a national network for ADR monitoring developed rapidly.3) Since 2003, a self-reporting internet-based monitoring system for spontaneous ADRs has operated.4) This greatly facilitates the collection of ADR information, and the number of adverse drug reactions reported has increased year-on-year since its introduction. The national ADR monitoring network received 1.429 million ADR or event reports in 2017 (1086 per million population).5) This far exceeds the WHO target of 200–400 per million population per year. ADR reports contain basic and clinical information about patients, and studies have shown that the completion of clinical information will affect the reports’ value.6,7) Incomplete and inaccurate information will seriously affect the evaluation of the relationship between a drug and a suspected ADR. It can even undermine the purpose of pharmacovigilance to “ensure the safe use of drugs.” Successful pharmacovigilance therefore requires not only a minimum number of ADR reports, but also good quality reports.
Studies have shown that the quality of ADR reports may need further improvements, for example, through integrity and standardization of the data, and accuracy of causal relationship evaluation.8) Regulatory agencies have noted that quality management systems are an essential component of successful pharmacovigilance practice.9) Improving the quality of ADR reports has therefore become an urgent issue in China. The purpose of this study was to analyze the quality of ADR reports in three provinces in east, central and west China (Jiangsu, Hubei and Shaanxi), and identify the factors affecting it.
China is a developing country with a large population and extensive territory. To provide a representative sample, we divided it into three regions: eastern, central and western. Combining the economic conditions, geographical location, population and medical level, we selected three provinces that were considered representative: Shaanxi province, Hubei province and Jiangsu province. In the meanwhile, researches in other similar fields were also carried out in Shaanxi, Hubei and Jiangsu province, as the representatives of western, central and eastern China.10–15) ADR reports collected from these three provinces during 2015–2017 were systematically sampled, then the sample was analyzed to identify the quality of ADR reports and the factors affecting it.
Sampling MethodThe sampling frame was all the ADR reports from the three provinces collected by the relevant central monitoring point from January 2015 to December 2017. The formula used to systematically sample the data was16):
 |
where P = sample variability, taking the maximum value of 0.5; e = the percentage accuracy of the survey results (5%); N = total sample size; and Z = 95% confidence interval (1.96).
Evaluation CriteriaThe sample reports were evaluated, assigned, sorted and counted using the items in the ADR report quality assessment system established by our research group.17) The scores for the 65 items in the reports were summed to provide a total score for each report (maximum 1000). The specific evaluation criteria have been shown in our previous research.17) The ADR reports were then separated into three quality classifications: high quality (score ≥800), medium quality (600≤ score <800) and poor quality (< 600).
Statistical AnalysisEach ADR report was classified in an Excel worksheet using the accuracy of each indicator. The quality of reports was compared by province, type of reports,18) reporting source and reporters’ occupation. The chi-square test (IBM-SPSS software version 13.0) was used to verify whether the differences were statistically significant. A value of p < 0.001 was considered to be significant.
In total, 3429 reports were included in the sample, with 1147 reports from Jiangsu, 1143 from Hubei, and 1139 reports from Shaanxi province. All these reports were included in the analysis. To evaluate the extraction of samples, the distribution of indicators was compared between the sample and all reports for Shaanxi province. The results show that the distribution of characteristics in the sample matched well with the overall situation (Table 1).
| Characteristics | 2015 (N = 36715, n = 379) | 2016 (N = 37459, n = 380) | 2017 (N = 37498, n = 380) | |||
|---|---|---|---|---|---|---|
| Na) (%) | nb) (%) | N (%) | n (%) | N (%) | n (%) | |
| Origin of the report | ||||||
| Medical institutions | 15914 (43.34) | 166 (43.80) | 10377 (27.70) | 95 (25.00) | 3594 (9.58) | 38 (10.00) |
| Pharmaceutical trading enterprises | 1269 (3.46) | 15 (3.96) | 1196 (3.19) | 11 (2.89) | 1018 (2.71) | 10 (2.63) |
| Manufacturing companies | 19352 (52.71) | 195 (51.45) | 25791 (68.85) | 273 (71.84) | 32848 (87.60) | 331 (87.11) |
| Patients | 180 (0.49) | 3 (0.79) | 95 (0.25) | 1 (0.26) | 38 (0.10) | 1 (0.26) |
| Type of the report | ||||||
| General | 29194 (79.52) | 299 (78.89) | 29625 (79.09) | 307 (80.79) | 27913 (74.44) | 300 (78.95) |
| New | 6875 (18.73) | 75 (19.79) | 6840 (18.26) | 59 (15.53) | 7640 (20.37) | 68 (17.89) |
| Serious | 646 (1.76) | 5(1.32) | 994 (2.65) | 14 (3.68) | 1945 (3.89) | 12 (3.16) |
| Gender | ||||||
| Male | 17666 (48.12) | 176 (46.44) | 17360 (46.34) | 191 (50.26) | 17073 (45.53) | 179 (47.11) |
| Female | 18919 (51.53) | 201 (53.03) | 19974 (53.32) | 187 (49.21) | 20322 (54.19) | 201(52.89) |
| Unknown | 130 (0.35) | 2 (0.53) | 125 (0.33) | 1 (0.26) | 103 (0.27) | 0 (0.00) |
a) Total number of ADR reports in each characteristic. b) The sampling ADR reports number in each characteristic.
Most ADR reports (2484, 72.44%) were classified as “general,” with “new” and “serious” reports making up 27.56% of the total. The number of “serious” reports was lowest, at just 187 (5.45%). The proportion of reports from medical institutions and pharmaceutical trading enterprises was 78.68% and 18.52%. These were the main sources of ADR reports. Pharmacists and doctors submitted more than 80% of the reports (Table 2).
| Characteristics | Quality classification of ADR reports | X2 | P | Total | ||
|---|---|---|---|---|---|---|
| High | Medium | Low | ||||
| Total number | 349 (10.18%) | 3017 (87.98%) | 63 (1.84%) | — | — | 3429 (100) |
| Year of the ADR reports | ||||||
| 2015 | 112 (9.8%) | 991 (86.8%) | 39 (3.4%) | 1142 (33.3) | ||
| 2016 | 93 (8.1%) | 1034 (90.5%) | 16 (1.4%) | 37.285 | <0.001 | 1143 (33.3) |
| 2017 | 144 (12.6%) | 992 (86.7%) | 8 (0.7%) | 1144 (33.4) | ||
| Geographical location | ||||||
| Jiangsu | 186 (16.2%) | 959 (83.6%) | 2 (0.2%) | 1147 (33.5) | ||
| Hubei | 147 (12.9%) | 986 (86.3%) | 10 (0.6%) | 209.302 | <0.001 | 1143 (33.3) |
| Shaanxi | 16 (1.4%) | 1072 (94.1%) | 51 (4.5%) | 1139 (33.2) | ||
| Type of the report | ||||||
| General | 44 (1.8%) | 2388 (96.1%) | 52 (2.1%) | 2484 (72.44) | ||
| New | 186 (24.5%) | 563 (74.3%) | 9 (1.2%) | 833.178 | <0.001 | 758 (22.11) |
| Serious | 106 (56.7%) | 79 (42.2%) | 2 (1.1%) | 187 (5.45) | ||
| Origin of the report | ||||||
| Medical institutions | 279 (10.3%) | 2386 (88.4%) | 33 (1.2%) | 2698 (78.68) | ||
| trading enterprises | 57 (9.0%) | 556 (87.6%) | 22 (3.5%) | 47.837 | <0.001 | 635 (18.52) |
| Manufacturing companies | 1 (1.1%) | 80 (89.9%) | 8 (9.0%) | 89 (2.60) | ||
| Patients | 1 (14.3%) | 6 (85.7%) | 0 (0.0%) | 7 (0.20) | ||
| Occupation of the reporter | ||||||
| Pharmacist | 171(11.5%) | 1311 (87.9%) | 10 (0.7%) | 1492 (43.51) | ||
| Doctor | 116 (9.2%) | 1115 (88.3%) | 32 (2.5%) | 58.137 | <0.001 | 1263 (36.83) |
| Nurse | 55 (10.7%) | 451 (87.7%) | 8 (1.6%) | 514 (14.99) | ||
| Othera) | 7 (4.4%) | 140 (87.5%) | 13 (8.1%) | 160 (4.67) | ||
a) includes personnel in the pharmacovigilance department of drug manufacturing companies and patients. Reporters in drug manufacturing companies are not usually healthcare professionals such as doctors, nurse and pharmacists.
Of the 3429 reports, 3017 (87.98%) were considered medium quality, 349 (10.18%) high quality and 63 (1.84%) low quality. There were statistically significant differences in the quality of ADR reports by year, province, report type, report source, and occupation of the reporter (chi-square test results show that p < 0.001) (Table 2). The quality of ADR reports was highest in 2017, and the percentage of low-quality reports decreased each year from 3.4% in 2015 to 1.4% in 2016 and 0.7% in 2017. The proportion of high-quality reports in each year was 9.8 8.1 and 12.6%.
Moving from the east to the west, the proportion of high-quality ADR reports decreased (16.2% in Jiangsu province, 12.9% in Hubei province and 1.4% in Shaanxi province), and the proportion of low-quality reports increased (0.2% in Jiangsu province, 0.6% in Hubei province and 4.5% in Shaanxi province). “New” and “serious” ADR reports were generally higher quality than “general” reports. In total, 57.6% of “serious,” 24.5% of “new,” and only 1.8% of “general” reports were high quality. Reports from doctors, pharmacists and nurses were higher quality than those from others: about 10% of the high quality reports and less than 3% of the low quality reports were from these groups, compared with 4.4% high quality and 8.1% low quality from other reporters. Reports from pharmacists were particularly good (11.5% high quality and 0.7% low quality). The quality of reports was higher from medical institutions and pharmaceutical trading enterprises than from other units, with about 10% high-quality and 3% poor-quality (Table 2).
The ADR reports contained eight major indicators and 56 specific items. Analyzing the quality by indicator category showed that report information, ADR analysis and reporting origin information were more complete and accurate than other categories. Of these three, report information and ADR analysis had the lowest percentage of missing information, close to zero (Table 3). The other five indicators, including patient information, drug information, ADR information, reporter information and vigilance, had more missing information. This is likely to be an important reason for the poor quality of ADR reports in China.
| Indicator/item | Missing information ratioa) (%) | Indicator/item | Missing information ratioa) (%) |
|---|---|---|---|
| 1) Report information | 4) ADR information | ||
| Report origin | 0.1% | ADR termination time | 45.30% |
| Code | 0.0% | Intervention ADR time | 34.10% |
| First time/tracking report | 0.0% | Normalization of ADR name | 27.70% |
| Report type | 0.0% | ADR sign description | 19.10% |
| 2) Patient information | ADR occurrence time | 15.60% | |
| Relevant important information | 78.50% | ADR intervention measures | 9.60% |
| Medical record number | 40.20% | ADR name | 2.40% |
| Body weight | 21.50% | Related clinical examination | 2.40% |
| Family adverse drug reactions | 18.60% | ADR symptom description | 1.60% |
| Past adverse drug reactions | 14.10% | 5) ADR analysis | |
| Nationality | 11.30% | Stopping/reducing drug response | 0.0% |
| Contact information | 9.40% | Re-drug response | 0.0% |
| Hospital name | 9.00% | Impact on the original disease | 0.0% |
| Original disease | 3.40% | Reporter evaluation | 0.0% |
| Name | 1.00% | Reporting unit evaluation | 0.0% |
| Gender | 0.30% | 6) Reporter information | |
| Date of birth/age | 0.00% | Reporter’s e-mail address | 94.10% |
| 3) Drug information | Reporter’s contact number | 1.20% | |
| Suspected drug–product name | 68.20% | Reporter occupation | 0.50% |
| Suspected drug–cause of medication | 18.90% | 7) Reporting origin information | |
| Suspected drug–approval number | 16.60% | Reporting origin contact number | 9.90% |
| Shared drug–product name | 8.50% | Reporting origin contact | 0.4% |
| Shared drug–cause of medication | 5.50% | Reporting origin name | 0.10% |
| Shared drug–usage approval number | 2.70% | Report date | 0.0% |
| Suspected drug–manufacturer | 1.50% | 8) Vigilance | |
| Suspected drug–common name | 1.40% | Reporting time limit | 12.0% |
| Shared drug–common name | 0.50% | ADR outcome | 1.8% |
| Suspected drug–usage dosage | 0.40% | Relevance rating | 1.3% |
| Suspected drug start and stop time | 0.40% | Impact on the original disease | 0.0% |
| Shared drug–manufacturer | 0.10% | ||
| Shared drug–usage dosage | 0.10% | ||
| Shared drug start and stop time | 0.10% |
a) The information missing from the report as a percentage of the whole.
Across the 56 items, we compared high quality items (those with less than 5% of information missing) and low quality items (those with more than 10% of information missing). The high quality items divided into two types: fundamental information such as NAME, CODE, and DATE, and fixed options such as REPORT TYPE, OCCUPATION, and ADR OUTCOME. The low quality items also divided into two categories: clinical specialty items and simple items that were poorly completed. The clinical specialty items showed two main issues. The first was an incomplete description of the ADR itself, such as ADR termination time, intervention time, and normalization of ADR name. The second was an unclear patient history related to ADRs, such as previous ADRs or ADRs in family members. The relatively simple items that were often poorly completed included medical record number, body weight, nationality, and reporter’s e-mail address. There were particularly high levels of missing information for reporter’s e-mail address, relevant important information and suspected drug-product name (deduction ratio >50%). These items should already have reduced the quality of ADR reports to some extent.
Three typical provinces were selected for this study: Jiangsu province, Hubei province and Shaanxi province (in eastern, central and western China). ADR reports from these provinces were sampled for the period from January 2015 to December 2017. The sample was compared to all reports in one province, and found to be representative. Only 10.18% of the 3429 ADR reports were considered to be high quality. This is similar to a study in France that found that only 12.7% of ADR reports from GPs were “well-documented.”19) However, it seems likely that the quality of ADR reports could be improved in China.
There were associations between report quality and year of the report, geographical location, type of the report, origin of the report, and occupation of the reporter (chi-square test, p < 0.01). The quality of ADR reports improved significantly during the three years studied, which is similar to results found in other studies.17,20) This may be because of increasing awareness of the importance of ADR monitoring, perhaps linked to a series of policies in China to promote national food and drug safety.9,21) Second, the provinces in eastern and central China showed better reporting quality. China is a large country with a relatively significant gap between rich and poor. Jiangsu and Hubei provinces have more developed economies and also more complete regulatory systems for ADR monitoring, which other studies have noted.22,23) Shaanxi province is a typical representative of the poorer western China provinces, and has worse supervision and policies on ADRs. It therefore seems likely that economic differences and mismatches in regional policy may be the main reasons for the geographical differences in report quality.
“New” and “serious” ADR reports were also higher quality than “general” reports. These reports may be considered more important for clinical drug use, and therefore the focus of the National Centre for ADR Monitoring. Reporters may therefore be more cautious in reporting and checking these reports. However, the number of “new” and “serious” ADR reports is small compared to “general” reports, making up less than 30% of the total. An Indian study found that 36.21% of ADR reports were considered to be serious.24) This suggests that there may be significant underreporting of serious ADRs in China. The same question has been raised in other studies.24–26) In recent years, China has adjusted some policies to reduce the underreporting of serious ADRs. For example, in 2015, 30% of all reports were expected to be “new” and “serious” ADRs as standard.27)
In our study, 78.68% ADR reports are from medical institutions. Among medical staffs, Pharmacists generally provided the highest quality ADR reports. According to Drug Administration Law of the People’s Republic of China,28) medical institutions, pharmaceutical trading enterprises and manufacturing companies have obligation to report ADR. What’s more, patients could report ADR voluntarily. In 2016, the Ministry of Health promulgated “the regulation on the administration of pharmaceutical affairs in medical institutions,” which stipulated the obligations of medical institutions and the responsibilities of pharmacists in the monitoring of adverse drug reactions.29) Pharmacists are usually responsible for ADR monitoring in medical institutions, and also participate in the formulation of clinical plans to guide clinical drug use. In recent years, some Chinese medicine hospitals have established a pharmacist-led ADR management model to promote hospital pharmacovigilance. Under this model, ADR reports are checked by pharmacists before they are submitted to the National Centre for ADR Monitoring. It is possible that pharmacists may take more responsibility for ADRs and report them more conscientiously.
In some countries, patients can report ADRs independently.30–32) Although patients can report ADRs independently in China, there is no clear system for them to do so. As a result, patients do not know how to report adverse drug reactions, and the proportion of ADRs reported by patients is low. There were 1317000 ADR reports in 2013, but only 0.6% were submitted by individuals, far lower than the 18% in Britain.33,34) In our study, only 0.2% of ADR reports were submitted by patients. This suggests that it may be necessary to reform the system for patients to report ADRs in China. Similarly, the number and quality of ADR reports from manufacturing companies was low. In developed countries, more than 80% of ADR reports are collected and reported by manufacturing companies, but in China, nearly 90% of ADRs were reported by medical institutions.35,36) Pharmaceutical manufacturers tend to keep away from ADR monitoring in China. On September 30, 2018, the Chinese state body for drug administration (SFDA) issued “a notice on the Direct Reporting of Adverse Drug Reactions (ADRs) by Marketing Authorization Holders (MAH),” which emphasized the responsibility of manufacturing companies for drug safety.37) Manufacturing companies should therefore develop effective measures to actively monitor and provide timely reports of any adverse reactions to their drugs.
The report categories of patient information, drug information, ADR information, reporter information and vigilance were less likely to be complete or correct, which affected the quality of ADR reports. Problems with patient information and reporter information may be caused by either a casual attitude from the reporters or the uncooperative behavior of patients. The reporters’ attitude is an important factor to ensure the success of pharmacovigilance. Some studies have suggested that a negative attitude among medical staff may lead to underreporting of ADR, and poor quality ADR reports.38–40) It is therefore necessary to publicize the importance of ADR monitoring to reporters, or perhaps develop some reasonable reward and punishment policies to encourage appropriate behaviors. One reason why patients may not cooperate is that they do not realize the importance of ADR monitoring, or lack trust in medical institutions. This idea was confirmed by a study exploring the determinants of public trust in the healthcare system in China.41) The patients who lacked trust in the healthcare system would generally not cooperate with medical staff or provide feedback on ADRs. The irregular completion of ADR information and drug information might be caused by the limited professional knowledge of the reporters. Patients may also not have the necessary information, because they did not realize that it would be considered important.
Some of the missing items were relatively simple, which suggests that reporters may not have realized that they were important. It may be helpful to include mandatory provisions in the report form so that reporters could only continue to report the ADR once these items had been completed. These items are relatively simple, so these mandatory provisions would not affect ADR reporting. Another more serious type of missing item was clinical specialty information. This information is important to evaluate causality and monitor drug safety. To ensure complete and accurate reporting of these items, the reporters need substantial clinical knowledge. However, some studies have suggested that reporters lack sufficient knowledge of ADR reporting and pharmacovigilance.42–44) We therefore suggest that hospitals, pharmaceutical trading enterprises and pharmaceutical manufacturing companies should train their pharmacovigilance personnel, including through lectures, brochures, and regular assessment. What’s more, for the missing drug information, such as suspected drug–product name, suspected drug-cause of medication and suspected drug-approval number, pharmaceutical manufacturing companies could print a QR code in the drug package, which would show all the standard drug information when user scan it. And if it is possible, we even could connect directly to the ADR monitoring system and an ADR report form with completed drug information would generated automatically by scanning the QR code. After completing other items in reporting form, the ADR report would be submit to the ADR monitoring center or other relative offices.
In conclusions, there are still some problems with the quality of ADR reports in China, although the quality of ADR reports improved year-on-year from 2015 to 2017. Reports from Jiangsu and Hubei provinces were better than those from Shaanxi province, and the quality of “new” and “serious” ADR reports was better than “general” reports. ADR reports from pharmacists were the highest quality. There were significant gaps in both some simple items that are easy to overlook and the clinical information in ADR reports. Effective measures to address these issues include improving the attitude of reporters to ADR monitoring, enhancing their knowledge of ADR reporting procedures, and increasing the cooperation of patients with ADR reporting.
The authors thank the three Centers for ADR monitoring in Shaanxi, Hubei and Jiangsu provinces.
The authors declare no conflict of interest.
