Circulation Journal
Online ISSN : 1347-4820
Print ISSN : 1346-9843
ISSN-L : 1346-9843
Rapid Communications
Quality Metrics for Heart Failure ― A Guideline-Driven Indicator Development Using a Modified Delphi Process and Its Applicability to Contemporary Japanese Practice ―
Takahiro SuzukiTakeshi KitaiTakashi KohnoShun KohsakaAtsushi Mizuno
Author information
JOURNAL OPEN ACCESS FULL-TEXT HTML

2025 Volume 89 Issue 4 Pages 509-515

Details
Abstract

Background: Evaluating the applicability of quality indicators (QIs) for heart failure (HF) care is crucial to addressing practice variability and improving outcomes by promptly translating guideline recommendations into QIs.

Methods and Results: A 12-member multidisciplinary panel assessed 35 HF care indicators from international guidelines using a Delphi survey (1–9 scale). Consensus (median ≥7) was achieved for 33 indicators (94.3%), but not for implantable cardioverter-defibrillator-related items. Device-related indicators revealed challenges in aligning international standards with the Japanese clinical context.

Conclusions: The study concluded that international HF QIs are generally applicable in Japan, though device-related QIs need further consideration.

Heart failure (HF) is a rapidly escalating global health challenge requiring complex therapeutic strategies and continual care optimization.1 Despite advancements in evidence-based treatments, substantial variations in the quality of HF care persist across healthcare institutions.2 Given these discrepancies, despite the widespread availability of proven therapies, implementing and systematically evaluating quality indicators (QIs) remain critical for establishing robust quality assurance, standardizing clinical practices, and facilitating ongoing improvements in HF management.3

Although international guidelines have provided recommendations for HF management, delayed adherence to QIs and performance measures (PMs) has resulted in delayed clinical feedback.4,5 The American College of Cardiology/American Heart Association (ACC/AHA) and the European Society of Cardiology (ESC) have developed comprehensive QIs for HF care to address this challenge. The ACC/AHA has updated PMs, advocating inpatient and outpatient care with Class I (strong) and Class III (harmful) recommendations.4 Similarly, the ESC has established QIs designed explicitly for contemporary HF care within European healthcare systems.5

Despite their widespread use in other Western countries, the applicability of these QIs in Japanese healthcare remains unclear, and QIs development within broader Asian contexts is likewise underexplored. The complexities of healthcare systems, patient demographics, clinical practices, cultural nuances, and resource availability can hinder direct extrapolation of externally-derived QIs to Japanese settings. Thus, in the present study we systematically assessed QIs for HF management based on international guidelines using a modified Delphi method to evaluate their appropriateness for Japanese healthcare, aiming to establish standard criteria, prioritize QI areas, and ultimately mitigate the HF burden in Japan.

Methods

Study Design

We conducted a modified Delphi consensus study in November 2022 using a web-based survey platform. To evaluate the QIs for HF care, we established a multidisciplinary expert panel comprising 12 members. The multidisciplinary panel was intentionally designed to reflect the team-based nature of HF care. We prioritized extensive clinical experience and representation from academic and community settings to ensure a comprehensive evaluation of the QIs across different practice environments. All panel members were chosen based on their significant clinical experience and active engagement in HF care, ensuring a holistic and practical approach to the evaluation process. Specifically, the panel included clinical cardiologists with extensive experience in HF, physical therapists specializing in rehabilitation for HF patients, general internists focusing on comorbid conditions management, nurses with expertise in HF patient education and care coordination, and pharmacists skilled in optimizing HF-related pharmacotherapy.

Selection of QIs

We initially reviewed the QIs proposed by the ACC/AHA, including PMs, quality measures (QMs), structural measures (SMs), rehabilitation PMs (Rehab PMs), and the ESC framework. Among 36 eligible QIs, we selected 35 indicators excepting ACC/AHA’s PM-11 (“Hydralazine/Isosorbide Dinitrate Therapy for HF With Reduced Ejection Fraction in Those Self-Identified as Black or African American”), reflecting its negligible presence in Japan.6 The selected QIs were organized into 6 narrative-based domains comprising “Assessment”, “Structure”, “Medical Intervention”, “Device (implantable cardioverter-defibrillator (ICD)/cardiac resynchronization therapy (CRT)”, “Rehabilitation” and “Follow-up”.

Modified Delphi Process

We used a structured modified Delphi method, using a 9-point Likert scale to evaluate each QI’s appropriateness, with 1 indicating minimal and 9 indicating maximum appropriateness. Panel members rated indicators based on their relevance to the Japanese healthcare context. Responses were collected anonymously through a secure online platform. QIs achieving a median score ≥7 were judged as consensus. Indicators below this threshold progressed to additional rounds, continuing until >90% of items reached consensus or response stability was observed.

Statistical Analysis

We summarized panel characteristics with descriptive statistics, calculating means for continuous variables such as years of experience, and proportions for categorical variables such as sex distribution. Regarding the Delphi method, we calculated median scores for each QI. We performed comprehensive descriptive statistical analyses to characterize the distribution of responses and evaluate the level of agreement. Furthermore, to clarify the discrepancies based on panelists’ backgrounds, we stratified analyses by physician vs. non-physician status and whether they had ≥15 years of clinical experience. The R software version 4.2.3, provided by the R Foundation for Statistical Computing, was used for all statistical analyses.

Results

We selected 12 panelists for the Delphi process, as summarized in Table 1. Among all panelists, 4 (33.3%) were female, and 8 (66.7%) had >15 years of clinical experience. The majority were 7 board-certified cardiologists (58.3%), with 1 general internist (8.3%), 2 nurses (16.7%), 1 pharmacist (8.3%), and 1 physical therapist (8.3%). Half of the panelists (50.0%) worked at university hospitals, and 1 (8.3%) worked in a general internal medicine clinic.

Table 1.

Characteristics of the Delphi Panelists

  No. (%)
Clinical discipline
 Cardiologist 7 (58.3)
 General internist 1 (8.3)
 Nurse 2 (16.7)
 Pharmacist 1 (8.3)
 Physical therapist 1 (8.3)
Sex, female 4 (33.3)
Years of clinical practice
 <15 4 (33.3)
 15–20 2 (16.7)
 20–25 1 (8.3)
 25–30 3 (25.0)
 >30 2 (16.7)
Type of working facility
 Hospital (university) 6 (50.0)
 Hospital (private hospital) 5 (41.7)
 Outpatient clinic 1 (8.3)

Table 2 summarizes the 35 QIs evaluated: 7 in “Assessment” (20.0%), 4 in “Structure” (11.4%), 12 in “Medical Intervention” (34.3%), 3 in “Rehabilitation” (8.6%), 4 in “Device (ICD/CRT)” (11.4%), and 5 in “Follow-up” (14.3%). Among them, 19 (54.3%) were unavailable in Japanese administrative data but could be accessed through electronic health records (EHR), and 4 (11.4%) were unavailable in both.

Table 2.

List of Quality Indicators in HF

Definition of quality indicator Reference Category Administrative data/
EHR data availability
LVEF assessment AHA PM-1 Assessment Both
Symptom and activity assessment AHA PM-2 Assessment EHR
Symptom management AHA PM-3 Assessment EHR
β-blocker therapy for HFrEF AHA PM-4 Medical intervention Both
ACE inhibitor, ARB, ARNI therapy for HFrEF AHA PM-5 Medical intervention Both
ARNI therapy for HFrEF AHA PM-6 Medical intervention Both
Dose of β-blocker therapy for HFrEF AHA PM-7 Medical intervention Both
Dose of ACE inhibitor, ARB, or ARNI therapy for HFrEF AHA PM-8 Medical intervention Both
MRA therapy for HFrEF AHA PM-9 Medical intervention Both
Laboratory monitoring in new MRA therapy AHA PM-10 Medical intervention EHR
Counseling regarding ICD implantation for patients with HFrEF on
guideline-directed medical therapy
AHA PM-12 Device (ICD/CRT) EHR
CRT implantation for patients with HFrEF on guideline-directed
medical therapy
AHA PM-13 Device (ICD/CRT) EHR
Patient self-care education AHA QM-1 Structure EHR
Measurement of patient-reported outcome-health status AHA QM-2 Follow-up None
Sustained or improved health status in HF AHA QM-3 Follow-up EHR
Post discharge appointment for patients with HF AHA QM-4 Follow-up EHR
HF registry participation AHA SM-1 Structure None
Exercise training referral for HF from inpatient setting AHA Rehab PM-2 Rehabilitation EHR
Exercise training referral for HF from outpatient setting AHA Rehab PM-4 Rehabilitation EHR
Centres should have a dedicated multidisciplinary team to manage
patients with HF
ESC Main (1.1) Structure None
Centres should have dedicated trained healthcare professionals to
deliver HF specific education to facilitate patient self-care
ESC Main (1.2) Structure None
Proportion of patients with HF who have a documentation of their
HF clinical type (HFrEF, HFmrEF, HFpEF)
ESC Main (2.1) Assessment EHR
Proportion of patients with HF who have a documentation of their
ECG findings
ESC Main (2.2) Assessment EHR
Proportion of patients with HF who have their NPs measured ESC Main (2.3) Assessment EHR
Proportion of patients with HF who have their blood tests
documented
ESC Main (2.4) Assessment EHR
Proportion of patients hospitalized with HF who have been referred
for a cardiac rehabilitation programme
ESC Main (2.5) Rehabilitation EHR
Proportion of patients hospitalized with HF who have a follow-up
review by a healthcare professional within 4 weeks of their hospital
discharge
ESC Secondary (2.1) Follow-up EHR
Proportion of patients with HFrEF who are prescribed the β-blocker
bisoprolol, carvedilol, sustained-release metoprolol succinate, or
nebivolol in the absence of any contraindications
ESC Main (3.1) Medical intervention Both
Proportion of patients with HFrEF who are prescribed an ACE
inhibitor, ARB or ARNI in the absence of any contraindications
ESC Main (3.2) Medical intervention Both
Proportion of patients with HFrEF who are prescribed a MRA in the
absence of any contraindications
ESC Main (3.3) Medical intervention Both
Proportion of patients with HFrEF who are prescribed a SGLT2
inhibitor in the absence of any contraindications
ESC Main (3.4) Medical intervention Both
Proportion of patients with HF who are prescribed loop diuretic
therapy if they have evidence of fluid retention
ESC Main (3.5) Medical intervention EHR
Proportion of symptomatic patients with HFrEF in sinus rhythm with
a QRS duration ≥150 ms and LBBB QRS morphology and with
LVEF ≤35% despite ≥3 months OMT who are offered CRT
ESC Secondary (4.1) Device (ICD/CRT) EHR
Proportion of symptomatic patients with HF, LVEF ≤35% despite ≥3
months of OMT, and IHD who are offered primary prevention ICD
ESC Secondary (4.2) Device (ICD/CRT) EHR
Proportion of patients with HF who have an assessment of their
HRQoL using a validated tool
ESC Secondary (5.1) Follow-up None

ACE, angiotensin-converting enzyme; AHA, American Heart Association; ARB, angiotensin-receptor blocker; ARNI, angiotensin receptor-neprilysin inhibitor; CRT, cardiac resynchronization therapy; ECG, electrocardiogram; EHR, electronic health record; ESC, European Society of Cardiology; HF, heart failure; HFmrEF, HF with mildly reduced ejection fraction; HFpEF, HF with preserved ejection fraction; HFrEF, HF with reduced ejection fraction; HRQoL, health-related quality of life; ICD, implantable cardioverter-defibrillator; IHD, ischemic heart disease; LBBB, left bundle branch block; LVEF, left ventricular ejection fraction; MRA, mineralocorticoid receptor antagonist; NPs, natriuretic peptides; OMT, optimal medical therapy; PM, performance measure; QM, quality measure; SGLT2, sodium-glucose co-transporter-2; SM, structural measure.

Among the 35 QIs evaluated, 33 (94.3%) met the median score criterion of ≥7 (Figure). All categories exceeded this threshold, except for 2 device-related items in “Device (ICD/CRT),” which scored <7: “Counseling regarding ICD implantation for patients with HF with reduced ejection fraction (HFrEF) on guideline-directed medical therapy” and “Proportion of symptomatic patients with HF, left ventricular ejection fraction (LVEF) ≤35% despite ≥3 months of optimal medical therapy (OMT), and ischemic heart disease (IHD) who are offered primary prevention ICD.” Consensus discrepancies were observed in device-related variables, as well as in areas such as LVEF assessment and mineralocorticoid receptor antagonist initiation, with physicians achieving the consensus while non-physicians fell below the threshold (Table 3).

Figure.

Delphi consensus of quality indicators for heart failure patient care illustrates the median (color bar) and mean (gray bar) agreement score on quality indicators using a 9-point Likert scale. ACE, angiotensin-converting enzyme; AHA, American Heart Association; ARB, angiotensin-receptor blocker; ARNI, angiotensin receptor-neprilysin inhibitor; CRT, cardiac resynchronization therapy; ECG, electrocardiogram; EHR, electronic health record; ESC, European Society of Cardiology; HF, heart failure; HFmrEF, HF with mildly reduced ejection fraction; HFpEF, HF with preserved ejection fraction; HFrEF, HF with reduced ejection fraction; HRQoL, health-related quality of life; ICD, implantable cardioverter-defibrillator; IHD, ischemic heart disease; LBBB, left bundle branch block; LVEF, left ventricular ejection fraction; MRA, mineralocorticoid receptor antagonist; NPs, natriuretic peptides; OMT, optimal medical therapy; PM, performance measure; QM, quality measure; SGLT2, sodium-glucose co-transporter-2; SM, structural measure.

Table 3.

Stratified Analysis of Delphi Consensus on HF Quality Indicators by Physician and Clinical Experience

Variable Physician Non-physician <15 years of
clinical experience
≥15 years of
clinical experience
Category
Median Mean Median Mean Median Mean Median Mean
AHA PM-1 9.0 8.3 5.5 6.0 8.0 8.0 7.5 7.3 Assessment
AHA PM-2 7.5 7.8 7.5 7.3 8.0 8.0 7.5 7.4 Assessment
AHA PM-3 7.0 7.1 5.5 5.8 7.0 7.3 6.5 6.4 Assessment
ESC Main (2.1) 9.0 8.9 8.5 8.0 9.0 9.0 9.0 8.4 Assessment
ESC Main (2.2) 8.5 8.0 7.5 7.3 8.5 8.0 7.5 7.6 Assessment
ESC Main (2.3) 9.0 8.4 7.5 7.3 8.0 7.8 9.0 8.1 Assessment
ESC Main (2.4) 8.5 8.3 7.5 7.3 8.5 8.0 8.0 7.9 Assessment
AHA QM-1 8.5 8.1 8.5 7.8 9.0 8.5 8.0 7.8 Structure
AHA SM-1 7.0 7.1 5.5 5.5 7.5 7.8 6.0 6.0 Structure
ESC Main (1.1) 9.0 8.5 7.5 6.8 8.5 8.5 8.5 7.6 Structure
ESC Main (1.2) 9.0 8.4 7.5 7.8 8.5 8.5 8.5 8.0 Structure
AHA PM-4 9.0 9.0 9.0 8.8 9.0 9.0 9.0 8.9 Medical
intervention
AHA PM-5 9.0 8.9 8.5 8.3 9.0 9.0 9.0 8.5 Medical
intervention
AHA PM-6 8.5 7.8 7.5 7.3 9.0 8.3 7.5 7.3 Medical
intervention
AHA PM-7 8.5 8.1 7.0 7.0 9.0 9.0 7.0 7.1 Medical
intervention
AHA PM-8 7.5 7.9 7.0 6.8 9.0 8.5 7.0 7.0 Medical
intervention
AHA PM-9 9.0 8.8 6.5 6.8 9.0 9.0 7.5 7.6 Medical
intervention
AHA PM-10 7.5 6.6 6.0 6.3 7.5 7.5 6.0 6.0 Medical
intervention
ESC Main (3.1) 9.0 8.9 9.0 8.8 9.0 9.0 9.0 8.8 Medical
intervention
ESC Main (3.2) 9.0 8.9 9.0 8.5 9.0 9.0 9.0 8.6 Medical
intervention
ESC Main (3.3) 9.0 8.8 8.0 8.0 9.0 9.0 9.0 8.3 Medical
intervention
ESC Main (3.4) 9.0 8.6 8.0 7.8 9.0 9.0 8.5 8.0 Medical
intervention
ESC Main (3.5) 8.0 7.6 8.0 7.5 9.0 8.5 7.0 7.1 Medical
intervention
AHA PM-12 7.0 7.3 4.5 5.5 7.5 7.5 6.5 6.3 Device
(ICD/CRT)
AHA PM-13 7.5 7.8 5.5 6.0 7.5 7.8 7.0 6.9 Device
(ICD/CRT)
ESC Secondary (4.1) 8.5 8.1 5.5 5.8 7.5 7.8 7.0 7.1 Device
(ICD/CRT)
ESC Secondary (4.2) 7.5 7.5 5.5 6.0 7.0 7.3 6.5 6.9 Device
(ICD/CRT)
AHA Rehab PM-2 8.5 8.4 8.5 7.8 8.5 8.5 8.5 8.0 Rehabilitation
AHA Rehab PM-4 7.0 6.8 8.0 7.5 7.0 6.5 7.0 7.3 Rehabilitation
ESC Main (2.5) 9.0 8.5 7.0 7.0 9.0 8.8 7.5 7.6 Rehabilitation
AHA QM-2 7.0 7.6 7.0 7.0 8.0 8.0 7.0 7.1 Follow-up
AHA QM-3 7.0 7.4 6.0 6.5 8.0 8.0 7.0 6.6 Follow-up
AHA QM-4 7.0 6.8 7.5 7.3 7.0 7.3 7.5 6.8 Follow-up
ESC Secondary (2.1) 8.5 8.1 8.5 7.8 9.0 8.8 7.5 7.6 Follow-up
ESC Secondary (5.1) 7.0 7.4 7.0 6.5 8.0 8.0 7.0 6.6 Follow-up

Abbreviations as in Table 2.

Discussion

This is the first study to systematically evaluate the applicability of international QIs for HF care in the Japanese healthcare context. The high consensus rate (94%) underscores the broad adaptability of these QIs despite certain nuances in the local context. Although international cardiology societies provide comprehensive HF guidelines, a persistent challenge remains in translating evidence-based recommendations into actionable metrics for quality assessment.4,5 This temporal gap in implementing guideline-based QIs highlights the importance of timely validation and ongoing re-evaluation of these measures. Our research provides a feasible framework for timely QI validation while emphasizing the necessity for third-party validation of guideline-derived metrics.

We observed strong consensus on patient management-related indicators (e.g., assessment, self-care education, rehabilitation, and follow-up) and medical intervention metrics (e.g., β-blocker and ACE inhibitor/ARB therapy). These findings align with previous studies that demonstrated adherence to guideline-recommended therapies improves outcomes in patients with HF with reduced EF (HFrEF).79 The robust agreement regarding follow-up care and cardiac rehabilitation mirrors evidence that structured programs reduce rehospitalization and enhance quality of life.10 Despite notable differences from other Western healthcare systems, our results suggest that international QIs can feasibly serve as benchmarks to drive quality improvement in Japan.

Our study revealed that 2 indicators in the “Device (ICD/CRT)” category did not achieve consensus: ICD counseling and primary prevention. Notably, when stratifying panelists by physician status, non-physicians showed less consensus than physicians regarding these indicators. One possible explanation is that the non-physician panelists, who often have fewer direct encounters with device implantation or follow-up management, may have a more limited understanding of the complexities involved in patient selection, cost-effectiveness, and potential complications in device-related therapy. Although the efficacy of ICDs in HFrEF has been well-established outside Japan,11 Japanese patients exhibit comparatively lower rates of sudden cardiac death. A contemporary registry reported an incidence of approximately 3% over 24 months, notably lower than that reported in non-Japanese populations.12 Moreover, Japanese patients who met the criteria of the MADIT-II trial, but did not receive ICD implantation, demonstrated mortality rates comparable to the defibrillator group in the MADIT-II trial and lower than those in the conventional therapy group.13 Consequently, adopting primary prevention ICDs remains low in Japan, with only 6.6% of eligible HFrEF patients receiving implantation.14 These findings underscore the need to carefully consider local epidemiology and clinical outcomes when defining QIs in Japan.

Our study also identified substantial challenges in data collection. Over half (54.3%) of the indicators could not be evaluated solely through administrative data, and an additional 11.4% were difficult to assess even with both administrative data and EHRs, suggesting the need for expanded data sources or indicator modification. Although the Japanese Circulation Society has established the Japanese Registry of All Cardiac and Vascular Diseases (JROAD) to capture patient characteristics and clinical data,15 administrative databases frequently lack the granularity needed to evaluate patient-reported outcomes, detailed resource utilization, and nuanced clinical endpoints.16 Consequently, integrating EHR data and other clinical registries is essential to achieve a more accurate and comprehensive assessment of HF care.17

Implementing QIs for HF in Japan faces ongoing hurdles related to limited resources, practice variability, disparities in device adoption, and provider-level barriers such as workforce shortages and cultural differences affecting shared decision-making. Although administrative data sources, particularly claims data, offer comprehensive coverage and longitudinal tracking capabilities, their reliability and clinical utility remain controversial among healthcare providers due to concerns about accuracy in complex clinical scenarios and the timeliness of updates. The varying technical capabilities across healthcare facilities in accessing and utilizing these databases present an additional layer of complexity. Nonetheless, advances in standardized data collection, broader administrative database access, and improved reporting mechanisms suggest growing feasibility for incorporating validated QIs into routine clinical practice. To maximize these opportunities, a systematic approach to QI implementation should focus on establishing reliable data collection methods, validating measurement tools, and developing infrastructure for consistent monitoring across different healthcare settings. Tailoring QIs to Japan’s healthcare context while leveraging these emerging data resources and improving multidisciplinary care coordination remain key strategies to overcome these barriers and enhance HF care quality.

Study Limitations

First, our validated QIs relied on expert opinions through a Delphi process, which may not fully reflect practical settings. Moreover, half of the panel members were cardiologists from university hospitals, potentially limiting the generalizability of our findings to primary care settings and rural regions. Limited representation from other healthcare professionals (i.e., only 1 pharmacist, 1 physical therapist, and 1 general internist) may have influenced consensus on multidisciplinary care indicators. Second, we did not distinguish between specific HF etiologies, such as ischemic and valvular heart disease, which could affect the applicability of some QIs in different HF subpopulations. Third, the small panel size might further constrain the generalizability of our results. Nevertheless, it adheres to the recommended panel size for Delphi studies, and our inclusion of experts with diverse clinical and research backgrounds helped strengthen the consensus process. Fourth, this study did not include the updated QI indicators for HF from the 2024 ACC/AHA and ESC guidelines, such as the use of SGLT2 inhibitors across the LVEF spectrum and intravenous iron therapy.18 As these newly added items were not fully evaluated, further studies are needed to assess their applicability. Additionally, the study focused on international guidelines without addressing Japan-specific indicators that could better reflect local healthcare practices. Fifth, we did not conduct a second consensus round. A modified Delphi process typically involves 2 rounds: in the first round, panelists independently rate the QIs, while the second round features a facilitated discussion to address disagreements and refine consensus. In our study, a second round was not undertaken because the panel only disagreed on device-related variables and showed strong agreement across all other domains. Given this high level of consensus and the minimal nature of unresolved disagreements, the panel concluded that the indicators were valid for all domains except device-related variables.

Conclusions

This study provides the first comprehensive evaluation of international QIs for HF care within the Japanese healthcare system. Experts’ consensus on most indicators was high, and the study identified specific challenges in device-related indicators and data collection methods. By placing the most appropriate QIs for the Japanese context, healthcare providers can better standardize and improve the quality of HF care. These insights can guide policymakers and healthcare administrators in implementing more effective quality measurement systems and improving the overall quality of HF care in Japan.

Funding

This study is based on the findings of the research program on “Human Resource Development for Data-Driven Healthcare Policy,” conducted at the Tokyo Foundation for Policy Research.

Authors’ Contributions

Data access and responsibility for integrity and accuracy: T.S., A.M. Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: T.S. Critical review of the manuscript: All authors. Supervision: S.K.

Declaration of Competing Interests

The authors have no financial disclosures or conflicts of interest to declare.

Acknowledgments

We thank all the panel members who contributed their time and expertise to participate in this consensus-building process.

IRB Information

As this study was part of hospital-wide quality improvement initiatives and involved no patient data, it was deemed exempt from institutional review board review per institutional policy.

Data Availability

The deidentified participant data will not be shared.

References
 
© 2025, THE JAPANESE CIRCULATION SOCIETY

This article is licensed under a Creative Commons [Attribution-NonCommercial-NoDerivatives 4.0 International] license.
https://creativecommons.org/licenses/by-nc-nd/4.0/
feedback
Top