Validity of Diagnostic Algorithms for Cardiovascular Diseases in Japanese Health Insurance Claims

Abstract

Background: We aimed to validate a claims-based diagnostic algorithm to identify hospitalized patients with acute major cardiovascular diseases (CVDs) from health insurance claims in Japan.

Methods and Results: This retrospective multicenter validation study was conducted at 4 institutes, including Japanese Circulation Society-certified and uncertified hospitals in Japan. Data on patients with CVDs in departmental lists or with International Classification of Diseases, 10th Revision (ICD-10) codes for CVDs hospitalized between April 2018 and March 2019 were extracted. We examined the sensitivity and positive predictive value of a diagnostic algorithm using ICD-10 codes, medical examinations, and treatments for acute coronary syndrome (ACS), acute heart failure (HF), and acute aortic disease (AAD). We identified 409 patients with ACS (mean age 70.6 years; 24.7% female), 615 patients with acute HF (mean age 77.3 years; 46.2% female), and 122 patients with AAD (mean age 73.4 years; 36.1% female). The respective sensitivity and positive predictive value for the algorithm were 0.86 (95% confidence interval [CI] 0.82–0.89) and 0.95 (95% CI 0.92–0.97) for ACS; 0.74 (95% CI 0.70–0.77) and 0.79 (95% CI 0.76–0.83) for acute HF; and 0.86 (95% CI 0.79–0.92) and 0.83 (95% CI 0.76–0.89) for AAD.

Conclusions: The validity of the diagnostic algorithm for Japanese claims data was acceptable. Our results serve as a foundation for future studies on CVDs using nationwide administrative data.

Cardiovascular diseases (CVDs) are a leading cause of death and medical expenditure in Japan, as well as worldwide.^1,2 The primary causes of acute hospitalization in cardiovascular departments are acute coronary syndrome (ACS), acute heart failure (HF), and acute aortic disease (AAD).³ The number of hospitalized patients with CVDs has increased in Japan because of the increasing aging population; however, few nationwide population-based cohort studies or studies on CVD outcomes have been conducted in Japan.

Administrative health data offer an opportunity to conduct population-level studies with a decreased risk of selection bias and improved generalizability.⁴ In addition, the use of administrative data for clinical research provides access to enormous amounts of real-world data at a low cost; therefore, administrative data are being increasingly used to examine practice patterns, quality of care, and health care utilization for a wide range of diseases.⁵

Previous studies in Japan have used large-scale administrative claims data, such as the National Database of Health Insurance Claims and Specific Health Checkups (NDB),⁶ which includes data on Japanese Diagnosis Procedure Combination (DPC) claims in addition to data on fee-for-service claims for both inpatients and outpatients.^7,8 However, descriptive and cohort studies on CVDs that estimate incidence and mortality rates using these databases are limited because they lack patient-level case validation. A prior single-center study reported that the positive predictive value (PPV) of the International Classification of Diseases, 10th Revision (ICD-10) codes for ACS in fee-for-service claims was low;⁹ therefore, more appropriate diagnostic algorithms are needed. Thus, we conducted a validation study to evaluate the validity of the diagnostic algorithms for 3 acute CVDs, namely ACS, acute HF, and AAD, using a combination of ICD-10 codes and procedures for CVDs.

Methods

Study Design and Population

This retrospective multicenter validation study of disease diagnosis of hospitalized patients focused on 3 major acute CVDs (ACS, acute HF, and AAD). We used health insurance claims data (Reseputo in Japanese) issued by Japanese hospitals. These data include Japanese DPC claims (Soukatsu reseputo) and fee-for-service claims (Ika reseputo; Supplementary Figure 1). This study was performed in accordance with the reporting guidelines for assessing the quality of validation studies using health administrative data.¹⁰

This study included all hospitalized patients aged ≥20 years with a clinical (based on chart review) or claims (based on ICD-10 and procedure codes) diagnosis for 1 of 3 major CVDs between April 1, 2018 and March 31, 2019. Patients with a clinical diagnosis of major CVDs were extracted based on each department’s list of all hospitalized patients at each hospital during the study period. After extracting data from the departmental lists of hospitalized patients in each division, cardiologists verified the clinical diagnoses based on the definitions of CVDs (a “positive case” was defined as chart review positive). Patients with CVD as a comorbidity only and with new-onset CVDs occurring after hospitalization were excluded. Patients with ICD-10 codes of major CVDs were extracted from the claims database of all hospitaized patients in each hospital, independently from the departmental lists (a “positive case” was defined as claims diagnosis positive). Because we did not review all the charts of patients who did not have ICD-10 codes of CVDs and were not included in the departmental lists, sensitivity was estimated assuming that there were no false negative cases in patients whose charts were not reviewed (Supplementary Figure 2).

We performed this study in 4 hospitals with cardiology departments in Osaka and Nara prefectures in Japan, namely Hospitals A, B, C, and D, which had 1,084, 992, 350, and 228 beds, respectively. All hospitals used the DPC system during the study period. Hospitals A and B are tertiary care hospitals in each prefecture, whereas Hospitals C and D are central hospitals in the secondary medical areas and in the urban and rural regions of Nara Prefecture. Hospitals A, B, and C were Japan Circulation Society (JCS)-certified training hospitals during the study period. In Hospitals A and B, coronary artery bypass grafting and surgery for AAD were performed; however, these procedures could not be performed in Hospitals C and D. Patients with the 3 acute CVDs were hospitalized in the Departments of Cardiology, Cardiovascular Surgery, or Radiology in Hospitals A and B; in the Department of Cardiology in Hospital C; and in the Departments of Cardiology or General Internal Medicine in Hospital D.

Reference Standard

The target conditions in this study were CVDs requiring emergency inpatient treatment. The 3 major CVDs were defined as follows:

• ACS included acute myocardial infarction (diagnosed as elevated levels of myocardial biomarkers [troponin or creatine kinase]) and unstable angina (diagnosed as new-onset, worsening, or resting chest pain with coronary stenosis but without elevated levels of myocardial biomarkers)^11,12

• Acute HF included HF diagnosed based on the Framingham criteria,¹³ which required emergency treatment;¹⁴ patients with end-stage renal disease who underwent hemodialysis or peritoneal dialysis were not considered to have a diagnosis of HF

• AAD included acute aortic dissection and aortic aneurysm rupture diagnosed by imaging (enhanced computed tomography [CT]) and requiring emergency hospitalization.¹⁵

Three experienced cardiologists not informed of the results of the algorithm (K.O., S.T., and K.K.) conducted a retrospective chart review to identify patients with any of the 3 CVDs.

Diagnostic Algorithm Using Claims Data

Because the PPV of ICD-10 codes alone was assumed to be low,⁹ diagnostic algorithms for CVDs were developed through discussion and consensus by the cardiologists (K.O., S.T., and K.K.) to improve diagnostic accuracy. The diagnostic algorithms consisted of a combination of ICD-10 codes, emergency admission status codes, treatment procedures, medications, and medical examinations (Figure). The ICD-10 codes included I20.0, I21, I22, and I23 for ACS; I50 and I11.0 for HF; and I71 and I72.3 for AAD (Supplementary Table).

Figure.

Diagnostic algorithms for (A) acute coronary syndrome (ACS), (B) acute heart failure (HF), and (C) acute aortic disease (AAD). CABG, coronary artery bypass grafting; CK-MB, creatine kinase MB; PCI, percutaneous coronary intervention.

The algorithm used the following criteria to diagnose ACS:

• The patient underwent percutaneous coronary intervention (PCI) or emergency coronary artery bypass grafting within 1 week after admission.

• The hospitalization was an emergency admission, the patients received antiplatelet therapy or heparin, and creatine kinase-MB was evaluated >3 times within 2 days of admission.

The algorithm diagnosed acute HF in cases in which there was a combination of emergency admission and intravenous diuretic use within 2 days after admission.

The algorithm diagnosed AAD if the following criteria were met:

• The patient underwent surgery with emergency admission, including open surgical or endovascular repair for aortic aneurysm or dissection.

• There was a combination of emergency admission with contrast-enhanced CT.

Statistical Analysis

The sensitivity (true positive/chart review positive) and PPV (true positive/chart review positive) of the diagnostic algorithm were calculated using the clinical diagnosis from chart reviews as the reference standard. We calculated 95% confidence intervals (CIs) for the binomial distributions using the exact method. The sensitivity and PPV were calculated according to the combination of the diagnosis: ICD-10 codes only, ICD-10 codes with emergency admissions, diagnostic algorithms, and ICD-10 codes with procedures (emergency PCI for ACS, intravenous diuretic use within 2 days of admission for acute HF, or surgical/endovascular repair within 1 week for AAD).

All statistical analyses were performed using STATA version 16 (StataCorp, College Station, TX, USA).

Patient and Public Involvement

Patients or the public were not involved in the study design, conduct, or reporting plans of this study.

Results

Overview and Patient Characteristics

We performed chart reviews of patients hospitalized with 1 of the 3 CVDs and patients with ICD-10 diagnosis codes for 1 of the 3 CVDs. Among patients diagnosed with CVDs through chart review, data were extracted for 409 patients with ACS, 615 patients with acute HF, and 122 patients with AAD. The mean (±SD) age of patients in the ACS, acture HF, and AAD groups was 70.6±11.9, 77.3±15.3, and 73.4±11.0 years, respectively. The proportion of women in the ACS, acute HF, and AAD groups was 24.7% (n=101), 46.2% (n=284), and 36.1% (n=44), respectively.

Validity for ACS

The frequency and validity of the ACS claims data are presented in Tables 1 and 2. A chart review identified 409 patients with ACS; 368 patients were identified as having ACS based on the diagnostic algorithm. In all hospitals, the algorithm sensitivity and PPV were 0.86 (95% CI 0.82–0.89) and 0.95 (95% CI 0.92–0.97), respectively. Although 140 hospitalized patients had ICD-10 codes for ACS, no patients were clinically diagnosed with ACS in Hospital D. All hospitals’ PPVs using the algorithm were >0.94, whereas the sensitivity was lower in Hospital A (0.79; 95% CI 0.66–0.88) than in Hospital C (0.94; 95% CI 0.87–0.98). Although the sensitivity of ICD-10 codes alone was higher than that of the algorithm, the PPV of ICD-10 codes alone was very low. Of all positive patients in the chart review (n=409), 39 (9.5%) did not have ICD-10 codes for ACS. The PPV of the ICD-10 codes plus procedure (emergency coronary intervention) was 1.0; however, the sensitivity was low compared with that of the algorithm.

Table 1. Validation of the Diagnostic Algorithms Based on the Claims

	Frequency (n)			SensitivityB (95% CI)	PPVC (95% CI)
	Chart review positiveA	Algorithm positive	True positive
Acute coronary syndrome
All hospitals	409	368	350	0.86 (0.82–0.89)	0.95 (0.92–0.97)
Hospital A	56	47	44	0.79 (0.66–0.88)	0.94 (0.82–0.99)
Hospital B	265	235	223	0.84 (0.79–0.88)	0.95 (0.91–0.97)
Hospital C	88	86	83	0.94 (0.87–0.98)	0.97 (0.90–0.99)
Hospital D	0	0	0	–	–
Acute heart failure
All hospitals	615	570	453	0.74 (0.70–0.77)	0.79 (0.76–0.83)
Hospital A	98	98	70	0.71 (0.61–0.80)	0.71 (0.61–0.80)
Hospital B	200	179	135	0.68 (0.61–0.74)	0.75 (0.68–0.82)
Hospital C	181	181	152	0.84 (0.78–0.89)	0.84 (0.78–0.89)
Hospital D	136	112	96	0.71 (0.62–0.78)	0.86 (0.78–0.92)
Acute aortic disease
All hospitals	122	126	105	0.86 (0.79–0.92)	0.83 (0.76–0.89)
Hospital A	47	61	47	1.00 (0.92–1.00)	0.77 (0.65–0.87)
Hospital B	64	57	51	0.80 (0.68–0.89)	0.89 (0.78–0.96)
Hospital C	9	7	6	0.67 (0.30–0.93)	0.86 (0.42–1.00)
Hospital D	2	1	1	0.50 (0.01–0.99)	1.00 (0.03–1.00)

^AChart review was used as the reference standard. ^BSensitivity was calculated as true positive/chart review positive. ^CThe positive predictive value (PPV) was calculated as true positive/algorithm positive. CI, confidence interval.

Table 2. Validation of the Combination of ICD-10 Codes and Other Treatments for Acute Coronary Syndrome

	Frequency (n)			SensitivityB (95% CI)	PPVC (95% CI)
	Chart review positiveA	Claims diagnosis positive	True positive
ICD-10 codes only
All hospitals	409	1,074	370	0.91 (0.87–0.93)	0.35 (0.32–0.37)
Hospital A	56	196	47	0.84 (0.72–0.92)	0.24 (0.18–0.31)
Hospital B	265	607	240	0.90 (0.86–0.93)	0.40 (0.35–0.44)
Hospital C	88	131	83	0.94 (0.87–0.98)	0.63 (0.54–0.72)
Hospital D	0	140	0	–	0 (0.00–0.03)
ICD-10 codes + emergency admission
All hospitals	409	744	340	0.83 (0.79–0.87)	0.46 (0.42–0.49)
Hospital A	56	99	47	0.84 (0.72–0.92)	0.48 (0.37–0.58)
Hospital B	265	447	212	0.80 (0.75–0.85)	0.47 (0.43–0.52)
Hospital C	88	110	81	0.92 (0.84–0.97)	0.74 (0.64–0.82)
Hospital D	0	88	0	–	0 (0.00–0.04)
Algorithm
All hospitals	409	368	350	0.86 (0.82–0.89)	0.95 (0.92–0.97)
Hospital A	56	47	44	0.79 (0.66–0.88)	0.94 (0.82–0.99)
Hospital B	265	235	223	0.84 (0.79–0.88)	0.95 (0.91–0.97)
Hospital C	88	86	83	0.94 (0.87–0.98)	0.97 (0.90–0.99)
Hospital D	0	0	0	–	–
ICD-10 codes + procedureD
All hospitals	409	311	311	0.76 (0.72–0.80)	1.0 (0.99–1.00)
Hospital A	56	26	26	0.46 (0.33–0.60)	1.0 (0.87–1.00)
Hospital B	265	209	209	0.79 (0.73–0.84)	1.0 (0.98–1.00)
Hospital C	88	76	76	0.86 (0.77–0.93)	1.0 (0.95–1.00)
Hospital D	0	0	0	–	–

^AChart review was used as the reference standard. ^BSensitivity was calculated as true positive/chart review positive. ^CThe positive predictive value (PPV) was calculated as true positive/claims diagnosis positive. ^DPercutaneous coronary intervention. CI, confidence interval; ICD, International Classification of Diseases, 10th Revision.

Validity for Acute HF

The frequency and validity of acute HF claims data are presented in Tables 1 and 3. After the chart review, 615 patients with acute HF were identified. Overall, 3,925 patients were identified as having acute HF based on ICD-10 codes; of these, 570 patients were identified using the diagnostic algorithm. In all hospitals, the sensitivity and PPV of the algorithm were 0.74 (95% CI 0.70–0.77) and 0.79 (95% CI 0.76–0.83), respectively. Although the sensitivity of the ICD-10 codes alone was higher than that of the algorithm, the PPV of the ICD-10 codes alone was extremely low. The PPV was higher for the algorithm and for ICD-10 codes plus procedure (intravenous diuretic use within 2 days of hospitalization) than for ICD-10 codes plus emergency admission. The PPVs of the algorithm were higher in Hospitals C and D than in Hospitals A and B. Of all positive patients in the chart review (n=615), 26 (4.2%) did not have ICD-10 codes for HF. Most patients who were clinically diagnosed with acute HF but did not fulfill the algorithm were patients who were only treated with oral diuretics (loop diuretics or tolvaptan) or other intravenous diuretics, such as carperitide; patients diagnosed with ACS; or patients with cardiogenic shock requiring catecholamine or mechanical support.

Table 3. Validation of the Combination of ICD-10 Codes and Other Treatments for Acute Heart Failure

	Frequency (n)			SensitivityB (95% CI)	PPVC (95% CI)
	Chart review positiveA	Claims diagnosis positive	True positive
ICD-10 codes only
All hospitals	615	3,925	589	0.96 (0.94–0.97)	0.15 (0.14–0.16)
Hospital A	98	1,756	91	0.93 (0.86–0.97)	0.05 (0.04–0.06)
Hospital B	200	836	190	0.95 (0.91–0.98)	0.23 (0.20–0.26)
Hospital C	181	672	177	0.98 (0.94–0.99)	0.26 (0.23–0.30)
Hospital D	136	661	131	0.96 (0.92–0.99)	0.20 (0.17–0.23)
ICD-10 codes + emergency admission
All hospitals	615	1,960	547	0.89 (0.86–0.91)	0.28 (0.26–0.30)
Hospital A	98	383	89	0.91 (0.83–0.96)	0.23 (0.19–0.28)
Hospital B	200	578	164	0.82 (0.76–0.87)	0.28 (0.25–0.32)
Hospital C	181	462	170	0.94 (0.89–0.97)	0.37 (0.32–0.41)
Hospital D	136	537	124	0.91 (0.85–0.95)	0.23 (0.20–0.27)
Algorithm
All hospitals	615	570	453	0.74 (0.70–0.77)	0.79 (0.76–0.83)
Hospital A	98	98	70	0.71 (0.61–0.80)	0.71 (0.61–0.80)
Hospital B	200	179	135	0.68 (0.61–0.74)	0.75 (0.68–0.82)
Hospital C	181	181	152	0.84 (0.78–0.89)	0.84 (0.78–0.89)
Hospital D	136	112	96	0.71 (0.62–0.78)	0.86 (0.78–0.92)
ICD-10 codes + procedureD
All hospitals	615	653	488	0.79 (0.76–0.82)	0.75 (0.71–0.78)
Hospital A	98	129	73	0.74 (0.65–0.83)	0.57 (0.48–0.65)
Hospital B	200	211	158	0.79 (0.73–0.84)	0.75 (0.68–0.81)
Hospital C	181	192	157	0.87 (0.81–0.91)	0.82 (0.76–0.87)
Hospital D	136	121	100	0.74 (0.65–0.81)	0.83 (0.75–0.89)

^AChart review was used as the reference standard. ^BSensitivity was calculated as true positive/chart review positive. ^CThe PPV was calculated as true positive/claims diagnosis positive. ^DIntravenous furosemide within 2 days after hospitalization. Abbreviations as in Table 2.

Validity for AAD

The frequency and validity of the administrative data for AAD are presented in Tables 1 and 4. Chart review identified 122 patients with AAD. Overall, 577 patients were identified as having AAD based on ICD-10 codes, whereas 126 were identified using the diagnostic algorithm. In all hospitals, the sensitivity and PPV of the diagnostic algorithm were 0.86 (95% CI 0.79–0.92) and 0.83 (95% CI 0.76–0.89), respectively. Although the sensitivity of the ICD-10 codes alone was higher than that of the algorithm, the PPV of the ICD-10 codes alone was very low. The PPV of ICD-10 codes plus procedure (surgical or endovascular repair) was 0.96%, and the sensitivity was lower (0.42%) than that of the algorithm. Of all positive patients in the chart review (n=122), 8 (6.6%) did not have ICD-10 codes for AAD. In most cases, patients who were clinically diagnosed with AAD but did not fulfill the algorithm for AAD were those who were diagnosed based on non-enhanced CT or those who had a cardiac arrest on admission. Most patients diagnosed using the algorithm but without a clinical diagnosis underwent enhanced CT for screening for anemia, to rule out AAD, or for preoperative screening of emergency heart surgery.

Table 4. Validation of the Combination of ICD-10 Codes and Other Treatments for Acute Aortic Disease

	Frequency (n)			SensitivityB (95% CI)	PPVC (95% CI)
	Chart review positiveA	Claims diagnosis positive	True positive
ICD-10 codes only
All hospitals	122	577	114	0.93 (0.87–0.97)	0.20 (0.17–0.23)
Hospital A	47	387	47	1.00 (0.92–1.00)	0.12 (0.09–0.16)
Hospital B	64	171	58	0.91 (0.81–0.96)	0.34 (0.27–0.42)
Hospital C	9	18	8	0.89 (0.52–1.00)	0.44 (0.22–0.69)
Hospital D	2	1	1	0.50 (0.01–0.99)	0.50 (0.01–0.99)
ICD-10 codes + emergency admission
All hospitals	122	215	106	0.87 (0.80–0.92)	0.49 (0.42–0.56)
Hospital A	47	77	47	1.00 (0.92–1.00)	0.61 (0.49–0.72)
Hospital B	64	123	51	0.80 (0.68–0.89)	0.41 (0.33–0.51)
Hospital C	9	14	7	0.78 (0.40–0.97)	0.50 (0.23–0.77)
Hospital D	2	1	1	0.50 (0.01–0.99)	0.50 (0.01–0.99)
Algorithm
All hospitals	122	126	105	0.86 (0.79–0.92)	0.83 (0.76–0.89)
Hospital A	47	61	47	1.00 (0.92–1.00)	0.77 (0.65–0.87)
Hospital B	64	57	51	0.80 (0.68–0.89)	0.89 (0.78–0.96)
Hospital C	9	7	6	0.67 (0.30–0.93)	0.86 (0.42–1.00)
Hospital D	2	1	1	0.50 (0.01–0.99)	1.00 (0.03–1.00)
ICD-10 codes + procedureD
All hospitals	122	53	51	0.42 (0.33–0.51)	0.96 (0.87–1.00)
Hospital A	47	20	20	0.42 (0.28–0.58)	1.00 (0.83–1.00)
Hospital B	64	33	31	0.48 (0.36–0.61)	0.94 (0.80–0.99)
Hospital C	9	0	0	0	–
Hospital D	2	0	0	0	–

^AChart review was used as the reference standard. ^BSensitivity was calculated as true positive/chart review positive. ^CThe PPV was calculated as true positive/claims diagnosis positive. ^DEmergency open surgical or endovascular repair. Abbreviations as in Table 2.

Discussion

In the present study we evaluated the validity of the diagnoses using health insurance claims data from 4 hospitals in Japan and chart review results as the reference standard. The sensitivity and PPV of the algorithms for ACS (0.86 and 0.95, respectively), acute HF (0.74 and 0.79, respectively), and AAD (0.86 and 0.83, respectively) were acceptable. The PPVs using the algorithm for the 3 diseases were higher than those of the ICD-10 codes alone or ICD-10 codes with emergency admission only. This is the first multicenter validation study conducted on 3 acute CVDs using Japanese health insurance claims data. The 4 participating facilities cover both urban and rural areas and include both JCS-certified and non-certified training hospitals in Japan. Our results will contribute to future research on CVDs using nationwide health insurance claims data in Japan.

Few validation studies on CVDs using health insurance claims data have been conducted in Japan. Most validation studies on an administrative database were performed using DPC data.^16–18 The DPC data include specific diagnosis categories, such as the main diagnosis, admission-precipitating diagnosis, and most resource-consuming diagnosis. The validity of the ICD-10 codes in specific diagnosis categories of the DPC data was high compared with those of the health insurance claims data.¹⁶ A previous single-center study reported that the PPV of ICD-10 codes for ACS in DPC data (0.89) was higher than that in health insurance claims data (0.27),⁹ and the diagnosis of myocardial infarction based on ICD-10 codes only in the health insurance claims database has been challenging. Although the validity of specific ICD-10 codes in DPC databases has been well validated, DPC databases do not cover all hospitals in Japan.^19,20 In the present study, we proposed diagnostic algorithms using a combination of ICD-10 codes and other treatment codes for acute CVDs. The sensitivity and PPV of our algorithms are reasonable, and an algorithm for identifying hospitalized patients with major acute CVDs can contribute to future research on health insurance claims data, such as that included in the NDB, which covers most of the population in Japan.²¹

The combination of ICD-10 codes, emergency admissions, and treatments improved the PPV with an acceptable decrease in sensitivity. A prior validation study for ACS and stroke demonstrated that the combination of ICD-10 codes with medications and procedures improved diagnosis validity.²² We found extremely low PPVs for ICD-10 codes only. Our results imply that the assumption of the number of patients with CVDs based on ICD-10 codes only can lead to an overestimation of the number of patients in real-world settings. In contrast, our results showed that a small number of true positive patients had no ICD-10 codes. Some of these patients had other ICD-10 codes related to the targeted ICD-10 codes, such as codes for valvular disease (I34 or I35) in patients with HF and codes for old myocardial infarction (I252) in those with ACS who had a past history of ACS. The PPV using specific treatments, such as PCI for ACS and emergency open surgical or endovascular repair for aortic AAD, was very high. However, the sensitivity was low compared with that of diagnostic algorithms because a certain number of patients were without specific treatments (e.g., Stanford Type B aortic dissection). Our algorithms have several components for detecting processes and may be complicated; however, these processes using medical examinations and procedures are necessary to ensure diagnostic accuracy.

The present study clarified that there are differences in the accuracy of the algorithms among CVDs. The sensitivity and PPV of the algorithm for acute HF were lower than the sensitivity and PPV of the algorithm for ACS. This is because acute HF is one of the common conditions accompanying other diseases, such as infectious diseases, and there are few specific treatments for acute HF. Although not examined in the present study, the use of medications, such as carperitide or tolvaptan, may have improved the algorithm PPV for acute HF. In addition, we found that the sensitivity and PPV of the algorithms differed by hospital. This study included various types of hospitals, and the baseline characteristics and treatments of patients may vary across hospitals.

Study Limitations

This study has several limitations. First, a chart review was conducted by a single cardiovascular specialist at each hospital, and the inter-rater agreement could not be assessed because of the large number of chart reviews. Experienced cardiologists performed the chart reviews in this study based on the definition for each CVD, and we consider that the diagnosis concordance is acceptable. Second, a limited number of patients may have been admitted to departments other than those examined in this study. We could not examine the patients hospitalized in other departments, possibly leading to an overestimation of the sensitivity in this study. To avoid misclassification, we reviewed the multiple divisions where patients with the target disease assumed to be hospitalized. We did not have the number of patients with ICD-10 codes who were not included in a department’s list in all hospitals because we did not distinguish whether the patient was in a department’s list at the time of chart review. However, we counted the number of true cases not in a department’s list in 3 hospitals (B, C, and D), and it was small (0–3 cases for each disease). This may indicate that the estimated sensitivity in the present study is near the “true” sensitivity. Third, we did not calculate the specificity and negative predictive value in this study. Fourth, we included only DPC hospitals in this study. Although both DPC and non-DPC hospitals have the same format of fee-for-service data, the practice patterns may differ between DPC and non-DPC hospitals. Fifth, although the cardiologists were blinded to the results of the algorithm, they were aware of some parts of the medical treatments used for the algorithm in this study. Finally, the accuracy of the diagnosis was only determined for hospitalized patients in Japan; therefore, we cannot use these algorithms for outpatient situations or apply them to other countries.

Conclusions

We evaluated the validity of diagnostic algorithms to identify ACS, acute HF, and AAD and found that these algorithms showed acceptable sensitivity and PPV in all hospitals. The diagnostic algorithms using a combination of ICD-10 codes, emergency admission, medical examinations, and treatments improved diagnostic accuracy. Our results will contribute to future research on CVDs using nationwide health insurance claims data in Japan.

Acknowledgments

None.

Sources of Funding

This work was supported by Labor Research Grants (Grant no. 19FA1002 and 22FA1701) from the Ministry of Health, Labour, and Welfare, Japan and Intramural Research Fund (22-6-5) for Cardiovascular diseases of National Cerebral and Cardiovascular Center.

Disclosures

The authors declare they have no conflict of interest with respect to this research study and paper. Y. Sakata is a member of Circulation Journal’s Editorial Team.

Author Contributions

K.K., the main author, contributed to the design of the project, data collection, and analysis, and wrote the draft of the manuscript. K.O. and S.T. contributed to data collection. D.K., T.S., and M.H. contributed to the study design and coordinated data collection. Y.I., Y.N., M.N., T.N., Y. Sakata, Y.M., Y. Saito, and T.I. contributed to the project design and supervised the study. All authors contributed to data interpretation, revised the draft, and reviewed and approved the final manuscript.

IRB Information

The study protocol was approved by the Ethics Committee of Nara Medical University (Registration no. 2847), as well as the ethics committees in each participating hospital. Informed consent was obtained in the form of opt-out, either on the hospital website or by notifying the hospital. This research was conducted in accordance with the Declaration of Helsinki.

Data Availability

Data cannot be shared for privacy or ethical reasons.

Supplementary Files

Please find supplementary file(s);

https://doi.org/10.1253/circj.CJ-22-0566

References

• 1.   Ministry of Health, Labour and Welfare. The Ministry of Health population movement statistics [in Japanese]. https://www.mhlw.go.jp/toukei/list/81-1.html (accessed February 14, 2022).
• 2.   Naghavi M, Abajobir AA, Abbafati C, Abbas KM, Abd-Allah F, Abera SF, et al. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980–2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet 2017; 390: 1151–1210, doi:10.1016/s0140-6736(17)32152-9.
• 3.   Kanaoka K, Okayama S, Nakai M, Sumita Y, Nishimura K, Kawakami R, et al. Hospitalization costs for patients with acute congestive heart failure in Japan. Circ J 2019; 83: 1025–1031, doi:10.1253/circj.CJ-18-1212.
• 4.   Snyder JM, Pawloski JA, Poisson LM. Developing real-world evidence-ready datasets: Time for clinician engagement. Curr Oncol Rep 2020; 22: 45, doi:10.1007/s11912-020-00904-z.
• 5.   Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol 2005; 58: 323–337, doi:10.1016/j.jclinepi.2004.10.012.
• 6.   Yasunaga H. Real world data in Japan: Chapter I NDB. Ann Clin Epidemiol 2019; 1: 28–30, doi:10.37737/ace.1.2_28.
• 7.   Kanaoka K, Soeda T, Terasaki S, Nishioka Y, Myojin T, Kubo S, et al. Current status and effect of outpatient cardiac rehabilitation after percutaneous coronary intervention in Japan. Circ Rep 2021; 3: 122–130, doi:10.1253/circrep.CR-20-0143.
• 8.   Kanaoka K, Iwanaga Y, Nakai M, Nishioka Y, Myojin T, Kubo S, et al. Outpatient cardiac rehabilitation dose after acute coronary syndrome in a nationwide cohort. Heart 2022; 109: 40–46, doi:10.1136/heartjnl-2021-320434.
• 9.   Ando T, Ooba N, Mochizuki M, Koide D, Kimura K, Lee SL, et al. Positive predictive value of ICD-10 codes for acute myocardial infarction in Japan: A validation study at a single center. BMC Health Serv Res 2018; 18: 895, doi:10.1186/s12913-018-3727-0.
• 10.   Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol 2011; 64: 821–829, doi:10.1016/j.jclinepi.2010.10.006.
• 11.   Kimura K, Kimura T, Ishihara M, Nakagawa Y, Nakao K, Miyauchi K, et al. JCS 2018 guideline on diagnosis and treatment of acute coronary syndrome. Circ J 2019; 83: 1085–1196.
• 12.   Collet JP, Thiele H, Barbato E, Barthélémy O, Bauersachs J, Bhatt DL, et al. 2020 ESC guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J 2021; 42: 1289–1367, doi:10.1093/eurheartj/ehaa575.
• 13.   McKee PA, Castelli WP, McNamara PM, Kannel WB. The natural history of congestive heart failure: The Framingham study. N Engl J Med 1971; 285: 1441–1446, doi:101056/nejm197112232852601.
• 14.   Tsutsui H, Isobe M, Ito H, Ito H, Okumura K, Ono M, et al. JCS 2017/JHFS 2017 guideline on diagnosis and treatment of acute and chronic heart failure: Digest version. Circ J 2019; 83: 2084–2184, doi:10.1253/circj.CJ-19-0342.
• 15.   Ogino H, Iida O, Akutsu K, Ueda H, Ohki T, Kaji S, et al. JCS/JSCVS/JATS/JSVS 2020 guideline on diagnosis and treatment of aortic aneurysm and aortic dissection [in Japanese]. https://www.j-circ.or.jp/cms/wp-content/uploads/2020/07/JCS2020_Ogino.pdf (accessed February 14, 2022).
• 16.   Yamana H, Moriwaki M, Horiguchi H, Kodan M, Fushimi K, Yasunaga H. Validity of diagnoses, procedures, and laboratory data in Japanese administrative data. J Epidemiol 2017; 27: 476–482, doi:10.1016/j.je.2016.09.009.
• 17.   Nakai M, Iwanaga Y, Sumita Y, Kanaoka K, Kawakami R, Ishii M, et al. Validation of acute myocardial infarction and heart failure diagnoses in hospitalized patients with the nationwide claim-based JROAD-DPC database. Circ Rep 2021; 3: 131–136, doi:10.1253/circrep.CR-21-0004.
• 18.   Fujihara K, Yamada-Harada M, Matsubayashi Y, Kitazawa M, Yamamoto M, Yaguchi Y, et al. Accuracy of Japanese claims data in identifying diabetes-related complications. Pharmacoepidemiol Drug Saf 2021; 30: 594–601, doi:10.1002/pds.5213.
• 19.   Kanaoka K, Okayama S, Nakai M, Sumita Y, Onoue K, Soeda T, et al. Number of cardiologists per cardiovascular beds and in-hospital mortality for acute heart failure: A nationwide study in japan. J Am Heart Assoc 2019; 8: e012282, doi:10.1161/jaha.119.012282.
• 20.   Terasaki S, Kanaoka K, Nakai M, Sumita Y, Onoue K, Soeda T, et al. Outcomes of catecholamine and/or mechanical support in Takotsubo syndrome. Heart 2022; 108: 1467–1473, doi:10.1136/heartjnl-2021-319904.
• 21.   Kubo S, Noda T, Myojin T, Nishioka Y, Higashino T, Matsui H, et al. National database of health insurance claims and specific health checkups of Japan (NDB): Outline and patient-matching technique. bioRxiv 2018, doi:10.1101/280008.
• 22.   Shima D, Ii Y, Higa S, Kohro T, Hoshide S, Kono K, et al. Validation of novel identification algorithms for major adverse cardiovascular events in a Japanese claims database. J Clin Hypertens (Greenwich) 2021; 23: 646–655, doi:10.1111/jch.14151.