Construction of Predictive Models for Cardiovascular Mortality by Machine Learning Approaches in Patients Who Underwent Transcatheter Aortic Valve Implantation

Abstract

Background: Prognostic models for cardiovascular death, but not all-cause death, after transcatheter aortic valve implantation (TAVI) have not been established yet.

Methods and Results: In 252 patients with aortic stenosis (AS) who underwent TAVI (men/women 83/169; mean age 85 years), we explored predictive models by machine learning for cardiovascular death using 62 candidates. During the follow-up period (mean 1,135 days), 13 (5.2%) patients died of cardiovascular disease. The least absolute shrinkage and selection operator (LASSO) feature selection identified 8 features as important candidates, including old myocardial infarction, triglycerides/high-density lipoprotein cholesterol (TG/HDL-C) ratio, Society of Thoracic Surgeons predicted risk of mortality score (STS-PROM), pulse rate, left atrium volume index, stroke volume index, estimated glomerular filtration rate, and albumin. Cox regression analyses with adjustment for age and sex showed that old myocardial infarction, high levels of TG/HDL-C, STS-PROM, and pulse rate, as well as low levels of glomerular filtration rate and albumin, were independent risk factors for cardiovascular death. Models of logistic regression (LR) and random survival forest (RSF) using the LASSO-selected features, except for STS-PROM, significantly improved predictive abilities for cardiovascular death compared with LR analysis using STS-PROM alone.

Conclusions: Machine learning models of prediction for cardiovascular death of LR and RSF using the LASSO-selected features are superior to a LR model using STS-PROM alone in patients with severe AS who underwent TAVI.

Aortic stenosis (AS) is a disease in which the aortic valve (AV) hardens and calcifies due to age-related changes, congenital bicuspid valve, and rheumatic fever. Chronic pressure overload of the left ventricle (LV) caused by AS leads to heart failure, resulting in pulmonary edema, low cardiac output and sudden death.¹ Regardless of symptoms, patients with very severe AS have a poor prognosis in the absence of therapeutic interventions for AS.² Transcatheter AV implantation (TAVI) has recently been developed to treat patients with severe AS who have high perioperative risks and are unsuitable for surgical AV replacement (SAVR).³ The Society of Thoracic Surgeons (STS) score is an indicator for perioperative short-term risk that is calculated by using 65 items based on a database of adult cardiac surgeries in the United States. Recently, the STS predicted risk of mortality (STS-PROM) score has been used worldwide for predicting outcomes in surgical treatments.⁴ The indication for TAVI had been limited to patients with severe AS who had a high risk for surgical death with an STS-PROM score ≥8%.⁵ However, since recent studies showed that mortality rates of TAVI and SAVR were comparable in patients with an STS-PROM score <4%,^6,7 the indication for TAVI has been expanding in patients with AS who have a relatively low surgical risk, leading to an annual increase in the number of TAVI procedures.⁸

One of the issues for the STS-PROM score is that perioperative risk is overestimated, especially in patients with AS who are scheduled to undergo TAVI.^9,10 In addition, the calculation formula of the STS-PROM score has not been disclosed, and the input of parameters is a time-consuming task. In contrast, it was previously reported that several factors, including age, sex, renal function, pulmonary disease comorbidities, activities of daily living and body mass index (BMI), were associated with short-term mortality (30 days–1 year) in patients with AS who underwent TAVI.^11–17 Furthermore, recent studies using machine learning (ML) have shown that multiple features (n=5–34) were selected as predictors for all-cause mortality during follow-up periods (30 days–5 years) in patients with AS who underwent TAVI (Supplementary Table 1). However, those studies did not focus on cardiovascular death as an outcome.

As models for accurate prediction of cardiovascular death in patients with severe AS who underwent TAVI using longitudinal data for a long follow-up period have not been sufficiently available, there is a need for widely and specifically adopted prognostic models for TAVI. In the present study, we investigated predictive models by ML for cardiovascular death during a relatively long follow-up period using various features including STS-PROM score in patients who underwent TAVI. Furthermore, predictive abilities of several ML models were compared.

Methods

The present study was a retrospective and single-center cohort study. This study conformed to the principles outlined in the Declaration of Helsinki and was approved by the institutional ethical committee of Sapporo Medical University (No. 312-215). Written informed consent was obtained from all patients.

Study Subjects and Clinical Endpoint

All patients with severe AS who underwent TAVI by transfemoral, transapical and trans subclavian approaches at Sapporo Medical University Hospital (Sapporo, Japan) during the period from January 1, 2015 to December 31, 2022 were enrolled in the present study (n=256).

Patients with dialysis, TAVI after SAVR, or TAVI-in-TAVI were not enrolled in the present study. A flowchart of the study participants is shown in Figure 1. Patients who did not receive follow-up examinations at least once until December 31, 2023 were excluded. After exclusion, a total of 252 patients (men/women 83/169; mean age 85 years) contributed to the present analyses. The main clinical endpoint was cardiovascular death caused by heart failure, myocardial infarction, stroke, arrythmia or sudden death. All-cause death was also checked. Cause of death was clinically diagnosed by attending cardiologists who were board-certified members of the Japanese Circulation Society.

Figure 1.

A flowchart of the study participants and machine learning analyses. Among 256 patients with severe aortic stenosis (AS) who underwent transcatheter aortic valve implantation (TAVI), a total of 252 patients (men/women 83/169) were recruited for analyses in the present study. To perform machine learning analyses, the recruited patients were divided into 2 groups, a training group (75%) and a test group (25%). IDI, integrated discrimination improvement; LASSO, least absolute shrinkage and selection operator; LR, logistic regression; NRI, net reclassification improvement; PFI, permutation feature importance; ROC, receiver operating characteristic curve; RSF, random survival forest; SHAP, SHapley Additive exPlanations; STS-PROM, Society of Thoracic Surgeons predicted risk of mortality score.

Measurements

Samples of urine and blood, and medical examinations were performed after overnight fasting. BMI was calculated as body weight in kilograms divided by height in meters squared. Blood pressure and pulse rate were measured on the arm. Estimated glomerular filtration rate (eGFR) was calculated using the following equation for Japanese people:¹⁸ eGFR (mL/min/1.73 m²) = 194 × serum creatinine^(−1.094) × age^(−0.287) × 0.739 (if female). Levels of total cholesterol (TC), high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C) and triglycerides (TG) were measured using enzymatic assays. As a surrogate indicator of the size of LDL particles, the ratio of TG and HDL-C (TG/HDL-C) was calculated. The non-HDL-C level was also calculated by subtracting the HDL-C level from the TC level. In addition, small dense LDL-C (sdLDL-C) was calculated by using 3-step Sampson’s equations.^19,20 The usefulness of estimated sdLDL-C has been confirmed in several studies.^21–23 The STS-PROM score was calculated by using 65 items on the website https://acsdriskcalc.research.sts.org/, as previously reported.⁴

Echocardiography examinations were performed using Vivid E-9 or Vivid E-95 (GE Healthcare Japan Corporation) by 6 well-experienced and certified echocardiographers with at least 5 years of experience who were trained by cardiologists. Left atrium (LA) volume (mL) and stroke volume (mL) were measured using biplane disk summation and the Doppler method, respectively. LA volume index (mL/m²) and stroke volume index (mL/m²) were calculated by adjustment of body surface area. Mild, moderate and severe reverse flows of aortic regurgitation, mitral regurgitation and tricuspid regurgitation were detected using Doppler color flow imaging.

LASSO Feature Selection

The least absolute shrinkage and selection operator (LASSO), a powerful algorithm used to perform regularization and feature selection, is useful when the number of predictive variables is larger than the number of events,²⁴ as in the present study. The method of LASSO is to penalize insufficient coefficients of linear regression to zero, leading to suppression of overfitting.²⁵ Following the standardization of predictors using the scale function with R software, LASSO feature selection was implemented via the glmnet R package (4.1-8). The optimal number of features was defined by lambda, which was calculated on the basis of minimizing partial likelihood, with the function cv.glmnet utilizing a 10-fold cross-validation.²⁶ Missing values were imputed by the median value of each parameter. Downstream analysis was performed using the non-zero coefficients with LASSO.

Construction of Models Including Logistic Regression and Random Survival Forest (RSF)

The recruited patients were divided into 2 groups, a training group (75%) and a test group (25%), with the scikit-learn (ver. 1.2.2) function in Python libraries using stratify parameters (Figure 1). Prediction for cardiovascular death was investigated by ML models including logistic regression (LR) using variables selected by LASSO feature selection (LR-LASSO) and RSF²⁷ using variables selected by LASSO feature selection (RSF-LASSO) with scikit-survival (ver. 0.20.0) in Python libraries. To determine optimal hyperparameters of the RSF model, 3-fold cross-validation with a grid-search (GridSearchCV) was performed to maximize the performance with suppressing the overfitting. In the present study, number of trees (n_estimators 160), maximum depth of the tree (max_depth 3), and minimum number of samples required to be at a leaf node (min_samples_leaf 6) were determined. All other parameters were set to the default settings.

Explainability for the RSF Model

It is difficult to adapt clinical settings in ML models including RSF due to the lack of transparency and explainability for their predictive output.²⁸ In the present study, the interpretation of selected variables in the RSF model was investigated using 2 metrics: permutation feature importance (PFI)²⁹ with the scikit-survival package, and SHapley Additive exPlanations (SHAP)³⁰ with the SHAP package (ver. 0.46.0). PFI was calculated using 10-times permutation as the mean with scikit-learn. SHAP values were calculated by SHAP explainer (shap.Explainer) using the predicted risk score by RSF predict function.

Statistical Analysis

Numeric variables are expressed as means±standard deviation (SD) for parameters with normal distributions and as medians (interquartile ranges) for parameters with skewed distributions. The distribution of each parameter was tested for its normality using the Shapiro-Wilk W test. Comparison between 2 groups was done with the Mann-Whitney’s U test. Intergroup differences in percentages of demographic parameters were examined using the chi-square test. The cumulative incidences of cardiovascular death and all-cause death were analyzed by Kaplan-Meier curves. The associations of cardiovascular death with parameters selected by the LASSO feature selection were investigated using Cox proportional hazard models with and without adjustment of confounders including age and sex. Hazard ratios (HR), 95% confidence intervals (CI) and Akaike’s information criterion (AIC) were calculated. Parameters with a lower AIC score constitute a better-fit model. To compare the discriminatory capacities of the LR model using the STS-PROM score alone, LR-LASSO and RSF-LASSO for predicting cardiovascular death during a follow-up period, the area under the curve (AUC) in receiver operating characteristic curve (ROC) analysis was investigated using the method of DeLong et al.³¹ Moreover, the different discriminatory values were examined using continuous net reclassification improvement (NRI) and integrated discrimination improvement (IDI) with those logistic models.^32,33 A P value <0.05 was considered statistically significant. All data were analyzed using Python 3.9.16 and R 4.3.1 (R Core Team, 2023).

Results

Characteristics of the Study Patients

Characteristics of the recruited patients at baseline are shown in Table 1. A total of 252 patients (men/women 83/169; mean age 85 years) was recruited, and 62 shown variables including physical examination, symptom, medical history, comorbidly, STS-PROM score, biochemical data, echocardiography data and TAVI procedure-related data were adopted in the ML models.

Table 1.

Characteristics of Patients With Aortic Stenosis Who Underwent TAVI at Baseline (n=252)

Item	Item no.	Value	Defect (n)
Age (years)	1	85±5	0
Sex, male	2	83 (32.9)	0
Body mass index	3	22.7±3.7	0
Body surface area (m2)	4	1.47±0.17	0
Systolic blood pressure (mmHg)	5	125±18	0
Diastolic blood pressure (mmHg)	6	67±12	0
Pulse rate (beats/min)	7	68±12	0
Current smoking habit	8	80 (31.7)	0
Symptom	–		–
NYHA class	9		0
I/II	–	174 (69.1)	–
III/IV	–	78 (30.9)	–
Chest pain	10	42 (16.7)	0
Syncope	11	19 (7.5)	0
Medical history	–		–
Cardiopulmonary arrest	12	3 (1.2)	0
Shock	13	6 (2.4)	0
Comorbidity	–		–
Diabetes	14	86 (34.1)	0
Dyslipidemia	15	161 (63.9)	0
Hypertension	16	202 (80.2)	0
Chronic kidney disease	17	155 (61.5)	0
Atrial fibrillation	18	82 (32.5)	0
Stroke	19	51 (20.2)	0
Coronary artery disease	20	96 (38.1)	0
Old myocardial infarction	21	11 (4.4)	0
Past surgical history	–		–
Percutaneous coronary intervention	22	84 (33.3)	0
Coronary artery bypass grafting	23	9 (3.6)	0
STS-PROM score	24	5.4 [4.0–8.2]	0
Biochemical data	–		–
Hemoglobin (g/dL)	25	11.2±1.6	0
Platelet (x104/μL)	26	17.1±55.0	0
Albumin (g/dL)	27	3.5±0.4	0
eGFR (mL/min/1.73 m2)	28	54.9±18.3	0
Hemoglobin A1c (%)	29	6.0±0.8	0
TC (mg/dL)	30	170±33	2
LDL-C (mg/dL)	31	93±27	0
HDL-C (mg/dL)	32	56±16	0
Non-HDL-C (mg/dL)	33	114±30	2
TG (mg/dL)	34	96 [69–132]	0
TG/HDL-C	35	1.8 [1.2–2.7]	0
sdLDL-C (mg/dL)	36	28 [22–35]	2
Echocardiography data	–		–
Interventricular septum thickness in diastole (mm)	37	10.9±1.6	0
Posterior LV wall thickness in diastole (mm)	38	10.6±1.6	0
LV internal dimension in diastole (mm)	39	42.0±6.0	0
LV internal dimension in systole (mm)	40	28.1±6.8	0
LV end-diastolic volume (mL)	41	80.4±9.4	1
LV end-systolic volume (mL)	42	33.5±21.2	1
LV ejection fraction (%)	43	60.7±8.4	0
Stroke volume index	44	38.7±9.1	0
LA volume index	45	47.2±18.1	1
AV area index	46	0.41±0.12	4
AV max. velocity (m/s)	47	4.4±0.9	0
Aortal annulus (mm)	48	21.8±1.8	13
Peak AV pressure gradient (mmHg)	49	79.7±29.7	0
Mean AV pressure gradient (mmHg)	50	47.4±18.9	0
Sinus of Valsalva (mm)	51	31.4±3.7	9
Sinotubular junction diameter (mm)	52	25.3±3.3	13
E/e′	53	18.2 [13.8–23.8]	4
Deceleration time (m/s)	54	268±97	2
Valvular heart disease	–		–
AR	55	130 (51.6)	0
MR	56	157 (62.3)	0
TR	57	160 (63.5)	1
Mean TV pressure gradient (mmHg)	58	25 [20–30]	11
TAVI procedure-related data	–		–
Approach	59		0
Transfemoral	–	237 (94.0)	–
Transapical	–	14 (5.6)	–
Transsubclavian	–	1 (0.4)	–
Operation time (min)	60	102 [80–125]	0
Valve size (mm)	61	26 [23–26]	0
Valve type	62		0
Balloon expandable	–	169 (67.1)	–
Self-expandable	–	83 (32.9)	–

Variables are expressed as n (%), mean±standard deviation or median [interquartile range]. AR, aortic regurgitation; AV, aortic valve; eGFR, estimated glomerular filtration rate; HDL-C, high-density lipoprotein cholesterol; LA, left atrium; LDL-C, low-density lipoprotein cholesterol; LV, left ventricle; MR, mitral regurgitation; NYHA, New York Heart Association; sdLDL-C, small dense LDL-C; STS-PROM, Society of Thoracic Surgeons predicted risk of mortality; TAVI, transcatheter aortic valve implantation; TC, total cholesterol; TG, triglycerides; TR, tricuspid regurgitation; TV, tricuspid valve.

Incidence Rates for Cardiovascular Death and All-Cause Death During the Follow-up Period

The mean follow-up period was 1,135±660 days (maximum 2,500 days), and follow-up summation was 286,171 person-days. A thrombosed valve and prosthetic valve endocarditis during the follow-up period were observed in 4 patients and 5 patients, respectively. There was no severe structural valve deterioration. Among the 252 patients, 71 (28.2%) patients died, and 13 (5.2%) patients died of cardiovascular disease including congestive heart failure (n=9), acute myocardial infarction (n=1), prosthetic valve endocarditis (n=1), ventricular fibrillation (n=1) and sudden death (n=1). Incidence rates for cardiovascular death and all-cause death were 0.5 per 10,000 person-days and 2.5 per 10,000 person-days, respectively (Supplementary Table 2).

Kaplan-Meier survival curves of cardiovascular death and all-cause death are shown in Figure 2A and Supplementary Figure 1 respectively. Cumulative incidences of cardiovascular death and all-cause death were 27.2% (95% CI 7.5–72.3) and 60.4% (95% CI 41.6–79.8), respectively.

Figure 2.

Cumulative incidence of cardiovascular death and least absolute shrinkage and selection operator (LASSO) feature selection. (A) Kaplan-Meier survival curves for cardiovascular death. (B) Top 8 features among 62 candidates with high importance determined by LASSO feature selection for cardiovascular death. eGFR, estimated glomerular filtration rate; LA, left atrium; STS-PROM, Society of Thoracic Surgeons predicted risk of mortality score; TG/HDL-C, triglycerides/high-density lipoprotein cholesterol.

Candidates in the LASSO Feature Selection

Among 62 variables as possible predictors for cardiovascular death, the top 8 features ordered by coefficients with 10-times cross-validation were identified by LASSO feature selection (Figure 2B). Old myocardial infarction, high levels of logarithmically transformed (Log) TG/HDL-C, Log STS-PROM score, pulse rate and LA volume index and low levels of stroke volume index, eGFR and albumin increased the predictive risk for cardiovascular death.

HRs for Cardiovascular Death and All-Cause Death by the Features Selected by LASSO

Results of Cox proportional hazards model analyses for cardiovascular death using each variable selected by LASSO feature selection are shown in Table 2. In unadjusted analyses, high levels of Log TG/HDL-C, Log STS-PROM score and pulse rate and low levels of eGFR and albumin were significant risk factors for cardiovascular death. LA volume index and stroke volume index tended to be risk factors for cardiovascular death. Results of HRs in adjusted analyses using age and sex were almost the same as those in unadjusted analyses. The AIC in the model using eGFR was lowest among the models in both unadjusted and adjusted analyses.

Table 2.

Cox Proportional Hazards Regression Analyses for Cardiovascular Death in Patients With Aortic Stenosis Who Underwent TAVI

	Unadjusted			Adjusted*
	HR (95% CI)	P value	AIC	HR (95% CI)	P value	AIC
Old myocardial infarction	4.87 (1.28–18.6)	0.021	122	4.87 (1.27–18.6)	0.021	126
Log TG/HDL-C (per 1)	4.15 (1.47–11.7)	0.007	118	4.21 (1.49–11.3)	0.005	122
Log STS-PROM score (per 1)	3.35 (1.29–8.71)	0.013	120	4.48 (1.57–12.8)	0.005	123
Pulse rate (per 1 beats/min)	1.05 (1.01–1.09)	<0.001	121	1.05 (1.01–1.09)	0.007	125
LA volume index (per 1 mL/m2)	1.02 (0.99–1.05)	0.064	124	1.02 (0.99–1.05)	0.065	126
Stroke volume index (per 1 mL/m2)	0.94 (0.89–1.004)	0.069	123	0.94 (0.89–1.004)	0.069	127
eGFR (per 1 mL/min)	0.93 (0.89–0.97)	0.001	113	0.93 (0.89–0.97)	0.001	117
Albumin (per 1 g/dL)	0.12 (0.03–0.45)	0.001	116	0.12 (0.03–0.44)	0.002	119

*Adjusted by age and sex. AIC, Akaike information criteria; CI, confidence interval; HR, hazard ratio. Other abbreviations as in Table 1.

For all-cause death, high levels of Log STS-PROM score and pulse rate, and low levels of LA volume index, stroke volume index and albumin were significant risk factors in the unadjusted and adjusted analyses (Supplementary Table 3).

Discriminatory Capacities of ML Models for Cardiovascular Death

ROC curve analyses for cardiovascular death showed that AUCs of LR using Log STS-PROM score alone, LR-LASSO and RSF-LASSO were 0.703 (95% CI 0.514–0.893), 0.895 (95% CI 0.840–0.949) and 0.928 (95% CI 0.868–0.988), respectively (Supplementary Figure 2). AUC of RSF-LASSO was significantly higher than that of LR using the Log STS-PROM score alone (P=0.028; Table 3). The values of IDI and NRI in both models of LR-LASSO and RSF-LASSO were significantly higher than those of LR using the Log STS-PROM score alone.

Table 3.

Discrimination of Machine Learning Models for Cardiovascular Death in Patients With Aortic Stenosis Who Underwent TAVI

Model	AUC		IDI		Category-free NRI
	Value (95% CI)	P value	Value (95% CI)	P value	Value (95% CI)	P value
LR-(STS-PROM)*	0.703 (0.514–0.893)	–	–	–	–	–
LR-LASSO**	0.895 (0.840–0.949)	0.058	0.123 (0.008–0.239)	0.036	0.841 (0.326–1.355)	0.001
RSF-LASSO***	0.928 (0.868–0.988)	0.028	0.164 (0.040–0.287)	0.009	0.958 (0.445–1.470)	<0.001

*Logistic regression (LR) using STS-PROM score alone. **LR using least absolute shrinkage and selection operator (LASSO)-selected 8 features, except for STS-PROM score. ***Random survival forest (RSF) using LASSO-selected 8 features, except for STS-PROM score. The LASSO-selected 8 features are as follows: old myocardial infarction, logarithmically transformed (Log) TG/HDL-C ratio, pulse rate, LA volume index, stroke volume index, eGFR, albumin and Log STS-PROM score. AUC, area under the curve; IDI, integrated discrimination improvement; NRI, net reclassification improvement. Other abbreviations as in Tables 1,2.

Explanations of Each Feature in the RSF-LASSO Model by Using PFI and SHAP

Analyses of PFI (Figure 3A) and SHAP (Figure 3B) showed global explanations for ranking of the 8 variables except for STS-PROM score selected by LASSO feature selection. In both PFI and SHAP analyses, eGFR was the most important feature for prediction of cardiovascular death, followed by albumin, Log TG/HDL-C, LA volume index, pulse rate and stroke volume index. Old myocardial infarction was not an important factor in both PFI and SHAP analyses.

Figure 3.

Explanations for outputs in the random survival forest (RSF) model using permutation feature importance (PFI) and SHapley Additive exPlanations (SHAP). (A) Global explanation of features by PFI analysis. (B) Global explanation of features by SHAP analysis. (C) Global explanations by a scatter plot for each feature with the corresponding SHAP value. Blue dots, individual patients; gray histograms, distribution of features. (D–G) Four examples of local explanations by a waterfall plot for each feature with the corresponding SHAP value. E[f(x)] and f(x) represent a predicted value at baseline and a predicted risk score by using a RSF model, respectively. The predicted risk was calculated by summing individual SHAP values from baseline. eGFR, estimated glomerular filtration rate; LA, left atrium; TG/HDL-C, triglycerides/high-density lipoprotein cholesterol.

In SHAP scatter plots, each point corresponds to a prediction for a single patient and provides the impact on the output of the RSF model by the SHAP value (Figure 3C). SHAP values were increased in patients with eGFR <40 mL/min/1.73 m², those with albumin <3.5 g/dL, those with Log TG/HDL-C >0.5, those with pulse rate >80 beats/min, and those with stroke volume index <30. There was a U-shaped curve of SHAP values for LA volume index.

As some examples of local explanations, the model’s outputs affected by selected variables with each SHAP value are shown in Figure 3D–G. The changes from predictive risk score (value 0.659) at baseline calculated by the SHAP algorithm are shown. In 1 case, the effect of a low level of albumin for increasing predictive risk was offset by the other normal values (Figure 3D). In another case, a low predictive risk was estimated because all of the variables were within normal ranges (Figure 3E). In the third case, cardiac dysfunction including a high LA volume index and a low stroke volume index increased the predictive risk (Figure 3F). In the last case, a high predictive risk was estimated because all of the variables were with abnormal ranges (Figure 3G).

Discussion

The present study showed that 8 features, including old myocardial infarction, Log TG/HDL-C, Log STS-PROM score, pulse rate, LA volume index, stroke volume index, eGFR and albumin, were identified by LASSO feature selection for cardiovascular death and that ML models of LR and RSF using the 8 variables except for Log STS-PROM score selected by LASSO feature selection can predict cardiovascular death with high accuracy in patients who underwent TAVI. Furthermore, predictive abilities for models of LR-LASSO and RSF-LASSO were superior to the LR model using the STS-PROM score alone. Those results suggest that ML models for prediction of cardiovascular death using variables selected by LASSO feature selection are useful in clinical practice and are superior to prediction using the STS-PROM score, a score that is a time-consuming task due to the calculation using 65 items.

There have been several longitudinal studies focused on prognosis in patients who underwent TAVI.^34,35 A meta-analysis using 24 studies investigating causes of death after 30 days from the TAVI procedure using the PubMed database from 2002 to 2014 showed that infection/sepsis (14.3%), heart failure (14.1%) and sudden death (10.8%) were the most common causes of death.³⁴ Another study, in which the causes of death in 3,434 patients who underwent TAVI in Denmark were investigated, showed that 1,254 (36.5%) patients died during a median follow-up period of 2.67 years and that the percentage of patients with cardiovascular death was 46.7% (n=586).³⁵ Furthermore, it has recently been reported that advanced heart failure (11.6%) and sudden cardiac death (7.5%) accounted for cardiovascular deaths during a median follow up of 2 years. As a further reduction in all-cause mortality rate is expected due to the expansion of TAVI indication for patients with a relatively low risk,^6,7 the establishment of predictive models for cardiovascular death is needed. However, in most previous studies using ML models that focused on prognosis after the TAVI procedure, all-cause death, but not cardiovascular death, was investigated as the primary endpoint (Supplementary Table 1). Therefore, in the present study, we mainly focused on cardiovascular death using ML models.

In previous studies that focused on prediction of all-cause death after the TAVI procedure by using several ML models, the numbers of selected features were 5–34 variables (Supplementary Table 1). There was only 1 study using LASSO for feature selection, showing 15 variables for candidates.³⁶ In the present study, among the LASSO-selected 8 features, Log TG/HDL-C, also known as the atherogenic index of plasma (AIP),³⁷ was developed as an indicator of the size of LDL particles³⁷ and has been proposed as a risk factor for atherosclerotic cardiovascular disease.³⁸ There has been no study in which the significance of AIP in patients with AS who underwent TAVI was investigated, and for the first time the present study showed that Log TG/HDL-C (AIP) was identified as an important risk factor for cardiovascular death in patients with underwent TAVI. As for the other selected features, myocardial infarction has been shown to induce necrosis of the myocardium and subsequent fibrosis, possibly leading to sudden death due to fatal arrhythmias and heart failure in both the acute and chronic phases of the disease.³⁹ Among the echocardiography data, a high LA volume index, a marker of volume overload, has been reported to be associated with a worse prognosis in the general population.⁴⁰ It has been reported that low-flow AS, indicated by a stroke volume index <35 mL/min, was a predictor of 5-year survival in patients with AS after SAVR.⁴¹ In addition, a high pulse rate,⁴² a low eGFR as a marker of renal dysfunction⁴³ and a low level of albumin as a marker of malnutrition⁴⁴ have been established as strong risk factors for cardiovascular mortality in previous clinical studies. Therefore, all of the features chosen by LASSO feature selection in the present study were reasonable as risk factors for cardiovascular death.

For construction of ML models, feature selection is an important step for predictive capability of an ML model and computational complexity.⁴⁵ When a large number of predictors and a small number of outcomes are used as in the present study, it is difficult to analyze them in classic statistical modeling. Even in such a situation, LASSO feature selection can extract effective variables involving postoperative prognosis with complex and multicollinearity relationships in relation to clinical settings.⁴⁵ In the present study, ML models of RSF-LASSO showed significant improvement of predictive capability for cardiovascular death after the TAVI procedure.

ML models usually lack transparency generating the outputs.⁴⁶ Regardless of good predictability, a potential problem of ML models including RSF is that an explanatory ability of covariate effects on survival cannot be provided.⁴⁷ To solve the issue, analyses of PFI and SHAP were performed in the present study. The principles of PFI and SHAP are based on a permutation and an approximated calculation for the Shapley value based on game theory,^29,30 respectively. Although both PFI and SHAP account for the feature importance and provide a global explanation of ML models, SHAP can describe contribution of prediction in each single feature and interactions between features within the dataset in addition to feature importance.⁴⁸ Interestingly, the calculated importance of old myocardial infarction was almost nothing in both PFI and SHAP in contrast to results of HRs in Cox regression analyses after adjustment of age and sex. Since the limited number of predictors (only a maximum of 3 variables) were subjected into multivariable Cox regression analyses due to a small number of cardiovascular deaths, the HR of old myocardial infarction might be overestimated. In contrast, RSF models in the present study were constructed by multiple explanatory variables after suppressing the overfitting, and SHAP also could reveal the interactions between those variables. Especially in a small number of events, ML models may be able to ensure a robust prediction of the outcome. Taken together, the results indicate that the combination of LASSO feature selection and an RSF model with post-hoc analyses of SHAP would be one of the best methods for prediction of cardiovascular death in patients who underwent TAVI.

Study Limitations

The present study has some limitations. First, the enrolled patients were Japanese only, at an advanced age from a single center, and the sample size was relatively small. In addition, since the number of cardiovascular deaths as a clinical endpoint was also small, only age and sex were adjusted in multivariable Cox regression analyses. Therefore, the possibility of selection bias and/or type 2 error cannot be ruled out. Second, since it is possible that prediction by ML models using a small dataset leads to overfitting, LASSO feature selection was applied to suppress the overfitting in ML models as previously described.⁴⁹ Third, unfortunately we could not obtain any datasets for external validation. Appropriate internal validation has recently been recommended to investigate ML models using all available data at the developmental stage of a predictive model rather than using external validation.^50,51 Therefore, an appropriate internal validation using k-fold cross validation was performed in the present study. However, transportability and generalizability in RSF models need to be validated by external datasets in the future. Fourth, several candidates selected as important features in previous studies, including race, medication and other indexes including EuroSCORE II and JapanSCORE, were not investigated in the present study. Last, as the cause of death was clinically diagnosed by attending physicians, a definitive diagnosis could not be made by autopsy.

Conclusions

ML models of LR and RSF using features chosen by LASSO feature selection, including old myocardial infarction, Log TG/HDL-C, pulse rate, LA volume index, stroke volume index, eGFR and albumin, predict cardiovascular death with high accuracy in patients with severe AS who underwent TAVI, and those models are superior to LR using the STS-PROM score alone. By building non-time-consuming ML models, more accurate prediction of cardiovascular death might be possible. It is also possible that the predictive model for cardiovascular risk is applied by using applications or a web service in the clinical setting.

Acknowledgments

We appreciate members of Sapporo Medical University Adiposcience Research Group (SMARG) for invaluable discussions.

Sources of Funding

M.T., Y.A. and M.F. were supported by JSPS KAKENHI (20K08913, 21K09181, 22K08313, 23K07993).

Disclosures

M.F. is a member of Circulation Reports’ Editorial Team. The other authors have no conflicts of interest to declare.

IRB Information

Name of the ethics committee: the Institutional Review Boards in Sapporo Medical University Hospital.

Supplementary Files

Please find supplementary file(s);

https://doi.org/10.1253/circrep.CR-24-0182

References

• 1.   Zheng KH, Tzolos E, Dweck MR. Pathophysiology of aortic stenosis and future perspectives for medical therapy. Cardiol Clin 2020; 38: 1–12.
• 2.   Kang DH, Park SJ, Rim JH, Yun SC, Kim DH, Song JM, et al. Early surgery versus conventional treatment in asymptomatic very severe aortic stenosis. Circulation 2010; 121: 1502–1509.
• 3.   Cribier A, Eltchaninoff H, Bash A, Borenstein N, Tron C, Bauer F, et al. Percutaneous transcatheter implantation of an aortic valve prosthesis for calcific aortic stenosis: First human case description. Circulation 2002; 106: 3006–3008.
• 4.   Shahian DM, Jacobs JP, Badhwar V, Kurlansky PA, Furnary AP, Cleveland JC Jr, et al. The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: Part 1–Background, design considerations, and model development. Ann Thorac Surg 2018; 105: 1411–1418.
• 5.   Mack MJ, Leon MB, Smith CR, Miller DC, Moses JW, Tuzcu EM, et al. 5-year outcomes of transcatheter aortic valve replacement or surgical aortic valve replacement for high surgical risk patients with aortic stenosis (PARTNER 1): A randomised controlled trial. Lancet 2015; 385: 2477–2484.
• 6.   Mack MJ, Leon MB, Thourani VH, Pibarot P, Hahn RT, Genereux P, et al. Transcatheter aortic-valve replacement in low-risk patients at five years. N Engl J Med 2023; 389: 1949–1960.
• 7.   Forrest JK, Deeb GM, Yakubov SJ, Gada H, Mumtaz MA, Ramlawi B, et al. 4-Year Outcomes of patients with aortic stenosis in the evolut low risk trial. J Am Coll Cardiol 2023; 82: 2163–2165.
• 8.   Carroll JD, Mack MJ, Vemulapalli S, Herrmann HC, Gleason TG, Hanzel G, et al. STS-ACC TVT registry of transcatheter aortic valve replacement. J Am Coll Cardiol 2020; 76: 2492–2516.
• 9.   Kofler M, Reinstadler SJ, Stastny L, Dumfarth J, Reindl M, Wachter K, et al. EuroSCORE II and the STS score are more accurate in transapical than in transfemoral transcatheter aortic valve implantation. Interact Cardiovasc Thorac Surg 2018; 26: 413–419.
• 10.   Stahli BE, Tasnady H, Luscher TF, Gebhard C, Mikulicic F, Erhart L, et al. Early and late mortality in patients undergoing transcatheter aortic valve implantation: Comparison of the novel EuroScore II with established risk scores. Cardiology 2013; 126: 15–23.
• 11.   Iung B, Laouenan C, Himbert D, Eltchaninoff H, Chevreul K, Donzeau-Gouge P, et al. Predictive factors of early mortality after transcatheter aortic valve implantation: Individual risk assessment using a simple score. Heart 2014; 100: 1016–1023.
• 12.   Seiffert M, Sinning JM, Meyer A, Wilde S, Conradi L, Vasa-Nicotera M, et al. Development of a risk score for outcome after transcatheter aortic valve implantation. Clin Res Cardiol 2014; 103: 631–640.
• 13.   Debonnaire P, Fusini L, Wolterbeek R, Kamperidis V, van Rosendael P, van der Kley F, et al. Value of the “TAVI2-SCORe” versus surgical risk scores for prediction of one year mortality in 511 patients who underwent transcatheter aortic valve implantation. Am J Cardiol 2015; 115: 234–242.
• 14.   Hermiller JB Jr, Yakubov SJ, Reardon MJ, Deeb GM, Adams DH, Afilalo J, et al. Predicting early and late mortality after transcatheter aortic valve replacement. J Am Coll Cardiol 2016; 68: 343–352.
• 15.   Edwards FH, Cohen DJ, O’Brien SM, Peterson ED, Mack MJ, Shahian DM, et al. Development and validation of a risk prediction model for in-hospital mortality after transcatheter aortic valve replacement. JAMA Cardiol 2016; 1: 46–52.
• 16.   Martin GP, Sperrin M, Ludman PF, de Belder MA, Redwood SR, Townend JN, et al. Novel United Kingdom prognostic model for 30-day mortality following transcatheter aortic valve implantation. Heart 2018; 104: 1109–1116.
• 17.   Yamamoto M, Otsuka T, Shimura T, Yamaguchi R, Adachi Y, Kagase A, et al. Clinical risk model for predicting 1-year mortality after transcatheter aortic valve replacement. Catheter Cardiovasc Interv 2021; 97: E544–E551.
• 18.   Matsuo S, Imai E, Horio M, Yasuda Y, Tomita K, Nitta K, et al. Revised equations for estimated GFR from serum creatinine in Japan. Am J Kidney Dis 2009; 53: 982–992.
• 19.   Sampson M, Ling C, Sun Q, Harb R, Ashmaig M, Warnick R, et al. A New Equation for calculation of low-density lipoprotein cholesterol in patients with normolipidemia and/or hypertriglyceridemia. JAMA Cardiol 2020; 5: 540–548.
• 20.   Sampson M, Wolska A, Warnick R, Lucero D, Remaley AT. A new equation based on the standard lipid panel for calculating small dense low-density lipoprotein-cholesterol and its use as a risk-enhancer test. Clin Chem 2021; 67: 987–997.
• 21.   Endo K, Kobayashi R, Tanaka M, Tanaka M, Akiyama Y, Sato T, et al. Validation of estimated small dense low-density lipoprotein cholesterol concentration in a Japanese general population. J Atheroscler Thromb 2024; 31: 931–952.
• 22.   Inyaku M, Tanaka M, Sato T, Endo K, Mori K, Hosaka I, et al. Calculated small dense low-density lipoprotein cholesterol level is a predominant predictor for new onset of ischemic heart disease. J Atheroscler Thromb 2024; 31: 232–248.
• 23.   Tanaka M, Sato T, Endo K, Inyaku M, Mori K, Hosaka I, et al. An increase in calculated small dense low-density lipoprotein cholesterol predicts new onset of hypertension in a Japanese cohort. Hypertens Res 2023; 46: 2635–2645.
• 24.   Tibshirani R. Regression shrinkage and selection via the lasso: A retrospective. J R Stat Soc Series B Stat Methodol 2011; 73: 273–282.
• 25.   McNeish DM. Using Lasso for predictor selection and to assuage overfitting: A method long overlooked in behavioral sciences. Multivariate Behav Res 2015; 50: 471–484.
• 26.   Engebretsen S, Bohlin J. Statistical predictions with glmnet. Clin Epigenetics 2019; 11: 123.
• 27.   Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random survival forests. Ann Appl Stat 2008; 2: 841–860.
• 28.   Moncada-Torres A, van Maaren MC, Hendriks MP, Siesling S, Geleijnse G. Explainable machine learning can outperform Cox regression predictions and provide insights in breast cancer survival. Sci Rep 2021; 11: 6968.
• 29.   Breiman L. Random forests. Machine Learning 2001; 45: 5–32.
• 30.   Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell 2020; 2: 56–67.
• 31.   DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 1988; 44: 837–845.
• 32.   Pencina MJ, D’Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 2011; 30: 11–21.
• 33.   Pencina MJ, D’Agostino RB, Pencina KM, Janssens AC, Greenland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol 2012; 176: 473–481.
• 34.   Xiong TY, Liao YB, Zhao ZG, Xu YN, Wei X, Zuo ZL, et al. Causes of death following transcatheter aortic valve replacement: A systematic review and meta-analysis. J Am Heart Assoc 2015; 4: e002096.
• 35.   Ostergaard L, Kober N, Petersen JK, Jensen AD, De Backer O, Kober L, et al. Long-term cause of death in patients who underwent transcatheter aortic valve implantation. Am J Cardiol 2023; 193: 91–96.
• 36.   Penso M, Pepi M, Fusini L, Muratori M, Cefalu C, Mantegazza V, et al. Predicting long-term mortality in TAVI patients using machine learning techniques. J Cardiovasc Dev Dis 2021; 8: 44.
• 37.   Dobiasova M, Frohlich J. The plasma parameter log (TG/HDL-C) as an atherogenic index: Correlation with lipoprotein particle size and esterification rate in apoB-lipoprotein-depleted plasma (FER(HDL)). Clin Biochem 2001; 34: 583–588.
• 38.   Rabiee Rad M, Ghasempour Dabaghi G, Darouei B, Amani-Beni R. The association of atherogenic index of plasma with cardiovascular outcomes in patients with coronary artery disease: A systematic review and meta-analysis. Cardiovasc Diabetol 2024; 23: 119.
• 39.   Berezin AE, Berezin AA. Adverse cardiac remodelling after acute myocardial infarction: Old and new biomarkers. Dis Markers 2020; 2020: 1215802.
• 40.   Hoit BD. Left atrial size and function: Role in prognosis. J Am Coll Cardiol 2014; 63: 493–505.
• 41.   Jonnala VR, Quadri HS, Pourafkari L, Fernandez SF, Iyer VS, Nader ND. Stroke volume index (SVI) predicts all-cause mortality following transcatheter aortic valve replacement (TAVR). Cardiovasc Revasc Med 2024, doi:10.1016/j.carrev.2024.06.014.
• 42.   Kannel WB, Kannel C, Paffenbarger RS Jr, Cupples LA. Heart rate and cardiovascular mortality: The Framingham Study. Am Heart J 1987; 113: 1489–1494.
• 43.   Coresh J, Turin TC, Matsushita K, Sang Y, Ballew SH, Appel LJ, et al. Decline in estimated glomerular filtration rate and subsequent risk of end-stage renal disease and mortality. JAMA 2014; 311: 2518–2531.
• 44.   Koertzen M, Punjabi P, Lockwood G. Pre-operative serum albumin concentration as a predictor of mortality and morbidity following cardiac surgery. Perfusion 2013; 28: 390–394.
• 45.   Premalatha M, Jayasudha M, Cep R, Priyadarshini J, Kalita K, Chatterjee P. A comparative evaluation of nature-inspired algorithms for feature selection problems. Heliyon 2024; 10: e23571.
• 46.   Handelman GS, Kok HK, Chandra RV, Razavi AH, Huang S, Brooks M, et al. peering into the black box of artificial intelligence: Evaluation metrics of machine learning methods. AJR Am J Roentgenol 2019; 212: 38–43.
• 47.   Pickett KL, Suresh K, Campbell KR, Davis S, Juarez-Colunga E. Random survival forests for dynamic predictions of a time-to-event outcome using a longitudinal biomarker. BMC Med Res Methodol 2021; 21: 216.
• 48.   Wang J, Sourlos N, Heuvelmans M, Prokop M, Vliegenthart R, van Ooijen P. Explainable machine learning model based on clinical factors for predicting the disappearance of indeterminate pulmonary nodules. Comput Biol Med 2024; 169: 107871.
• 49.   Kamran H, Tahir M, Tayara H, Chong KT. iEnhancer-Deep: A computational predictor for enhancer sites and their strength using deep learning. Appl Sci 2022; 12: 2120.
• 50.   Collins GS, Dhiman P, Ma J, Schlussel MM, Archer L, Van Calster B, et al. Evaluation of clinical prediction models (part 1): From development to external validation. BMJ 2024; 384: e074819.
• 51.   Shyalika C, Wickramarachchi R, El Kalach F, Harik R, Sheth A. Evaluating the role of data enrichment approaches towards rare event analysis in manufacturing. Sensors 2024; 24: 5009.