Validation of the Korean Genome Epidemiology Study Risk Score to Predict Incident Hypertension in a Large Nationwide Korean Cohort

Abstract

Background: This study aimed to validate the Korean Genome Epidemiology Study (KoGES) risk score to predict the 4-year risk of hypertension (HT) in a large nationwide sample, and compare its discrimination and calibration with the Framingham and blood pressure (BP)-only models.

Methods and Results: This study analyzed 69,918 subjects without HT at baseline from the National Sample Cohort in the National Health Insurance Service database. We compared the Framingham, KoGES, and BP-only models for discrimination using area under the receiver-operating characteristic curves (AROC), calibration using goodness-of-fit tests, and reclassification ability using the continuous net reclassification improvement (NRI) and integrated discrimination improvement. Of 69,918 subjects, 18.6% developed HT during the follow-up. AROC was significantly higher for the KoGES (0.733) than for the Framingham (0.729) or BP-only (0.707) model. Recalibrated Framingham model underestimated HT incidence in all deciles (P<0.001). BP-only model overestimated risk in the lower deciles (P<0.001). KoGES model accurately predicted risk in all except the highest decile (χ²=14.85, P=0.062). The KoGES model led to a significant improvement in risk reclassification compared with the Framingham and BP-only models (NRI, 0.354; 95% confidence interval [CI], 0.343–0.365 and 0.542; 95% CI, 0.523–0.561, respectively).

Conclusions: In this validation study, the KoGES model demonstrated better discrimination, calibration, and reclassification ability than either the Framingham or BP-only model. The KoGES model may help identify Korean individuals at high risk for HT.

Hypertension (HT) is a major risk factor for cardiovascular disease (CVD), stroke, and kidney disease.^1–4 Worldwide, HT is responsible for over 45% and 51% of heart disease and stroke mortalities, respectively.⁵ Therefore, identifying individuals at high risk for incident HT may guide implementation of preventative lifestyle modifications or treatment programs.

Many randomized controlled trials and observational studies of high-risk individuals show that lifestyle modification or treatment can help prevent or delay the incidence of HT.^6–9 Recently, many risk prediction scores based on multiple cohort studies have been developed in Caucasian and Asian populations.^10–14 However, most of these risk scores have not been calibrated or externally validated in different ethnic populations.¹⁵ The accuracy of the risk score for predicting HT determines its clinical utility among physicians.¹⁵ However, application of risk scores in different ethnicities or nations without external validation may result in inaccurate estimation of the risk for chronic disease.^16–18

In 2013, an accurate HT risk score was developed using a large community-based cohort of middle-aged Korean individuals from the Korean Genome Epidemiology Study (KoGES).¹³ The objectives of the present study were to validate the KoGES in a large, nationwide sample cohort in Korea and compare its discrimination, calibration, and reclassification ability with the recalibrated Framingham risk score and blood pressure (BP)-only models.

Methods

Data Source

This study used the National Sample Cohort (NSC), which included data of socioeconomic variables, and details of medical treatment and health examinations from approximately 1 million people from 2002 to 2013. Approximately 2% of the entire Korean population was extracted from a database of the National Health Insurance Service using a stratified random sampling method. The NSC consisted of eligibility, medical treatment, health examination, and medical care institution databases, and has been described in detail previously.¹⁹ The medical treatment database included details of medical treatment, disease codes, and prescription. The health examination database contained various health lifestyles and behaviors obtained from self-reported questionnaires, and results of biochemical and anthropometric tests. All diagnostic codes were filled using the Korean Classification of Disease, 5th edition, which was modified from the International Classification of Disease-10. The prescription details included the brand and generic names of the drugs, and the date and duration of the prescription.

Study Population

We enrolled 130,569 subjects aged 40–69 years who had undergone a health examination in 2009. To identify patients with incident HT, we excluded those who did not attend at least 1 follow-up health examination in 2011 or 2013 (n=29,690), with HT (n=30,708) at baseline in 2009, with other CVDs (n=192), or with incomplete covariate data (n=61). Finally, 69,918 subjects were selected for analysis.

Measurements and Surveys

HT was defined as systolic BP ≥140 mmHg or diastolic BP ≥90 mmHg on health examination, or a record with hypertensive disease codes (I10–I13) and prescription of one of the following antihypertensive agents: angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers, β-blockers, calcium-channel blockers, diuretics, or other antihypertensive drugs. Body mass index (BMI) was calculated by dividing the weight in kilograms by the height in meters squared. Current smoking status and parental history of HT were classified based on the self-reported questionnaire.

Statistical Analysis

The KoGES and Framingham models have the same risk variables and interaction terms as follows: age, sex, current smoking, BMI, parental HT, systolic and diastolic BPs, interaction of diastolic BP and age. The estimated 4-year risk (P̂) for the incidence of HT by the KoGES and Framingham models can be calculated by following specific equation using estimated β coefficients (βs) and scale parameter (σ) by Weibull regression analysis.^10,13

Additionally, we also derived a simple model, a BP-only model that included systolic and diastolic BPs, to compare the performance between 3 models. The exact equations used to calculate risk for HT with each of the 3 models are shown in the Appendix.

The parental history of HT was evaluated using 2 categories (1: yes or 0: no) in the validation cohort and 3 categories (neither, one, or both) in the Framingham and KoGES models. To directly apply the regression coefficients of both models to the NSC data, we calculated weighted averages of the distribution of the parental history of HT in the Framingham and KoGES studies.^10,11,13 The BP-only model, which used systolic and diastolic BPs, was constructed using multiple logistic regression analysis. We compared discrimination using area under the receiver-operating characteristic curves (AROCs) according to the method of DeLong et al and calibration using the goodness-of-fit test procedure based on Hosmer and Lameshow’s chi-square test among the Framingham, KoGES, and BP-only models.^20,21 We assessed reclassification ability using the continuous and user-categorized net reclassification improvement (NRI) and integrated discrimination improvement (IDI). A 2-tailed P-value <0.05 was considered statistically significant. Statistical analyses were performed using SAS software version 9.3 (SAS Institute, Cary, NC, USA).

Results

Of the 69,918 subjects, 18.6% developed HT during the 4-year follow-up period. The baseline characteristics of the validation cohort with incident HT are presented in Table 1. Mean age of the validation cohort was 50.3 years and 49.5% were women. A greater proportion of subjects with incident HT were male (58.4% vs. 41.4%, P<0.001). Age, BMI, and the proportion of current smokers were higher among subjects with HT.

Table 1. Baseline Characteristics of the HT Risk Score Validation Cohort

Variable	Total (n=69,918)	Normotensive (n=56,940)	Hypertensive (n=12,978)	P value†
n, %	100.0	81.4	18.6
Sex, women, %	34,640 (49.5)	51.4	41.3	<0.0001
Age, mean (SD), years	50.3 (7.5)	49.7 (7.3)	52.6 (7.9)	<0.0001
BP, mean (SD), mmHg
Systolic	118.1 (11.3)	116.7 (11.2)	124.3 (9.7)	<0.0001
Diastolic	73.9 (8.0)	73.0 (7.9)	77.8 (6.9)	<0.0001
Current smoking, %	21.3	20.7	24.0	<0.0001
Parental HT, %	8.8	8.4	10.6	<0.0001
BMI, mean (SD), kg/m2	23.5 (2.8)	23.3 (2.7)	24.4 (2.9)	<0.0001

^†P-value for χ² or t-test procedure. BMI, body mass index; BP, blood pressure; HT, hypertension; SD, standard deviation.

The AROC was 0.733 for KoGES, 0.729 for the recalibrated Framingham, and 0.707 for the BP-only model (Figure 1). The AROC was significantly higher in the KoGES model than in the recalibrated Framingham model (∆=0.004; 95% confidence interval [CI], 0.003–0.005) and the BP-only model (∆=0.026; 95% CI, 0.024–0.028) (Table 2).

Figure 1.

Receiver-operating characteristic curves for prediction of hypertension incidence using the Korean Genome Epidemiology Study (KoGES), Framingham, and blood pressure-only models. Respective areas under the receiver-operating characteristic curves are 0.707, 0.729, and 0.733.

Table 2. Comparison of Discrimination and Reclassification in KoGES, Recalibrated Framingham, and BP Only Model

Measurement	KoGES vs. Framingham recalibrated	P value	KoGES vs. BP only	P value
ΔAROC (95% CI)	0.004 (0.003–0.005)	<0.001	0.026 (0.024–0.028)	<0.001
NRI (95% CI)	0.354 (0.343–0.365)	<0.001	0.400 (0.382–0.418)	<0.001
IDI (95% CI)	0.023 (0.022–0.024)	<0.001	0.029 (0.027–0.030)	<0.001

AROC, area under the receiver operating characteristic curve; BP, blood pressure; CI, confidence interval; IDI, integrated discrimination improvement; NRI, continuous net reclassification improvement.

Figure 2 shows the agreement between the observed risk and mean predicted risk estimated by the 3 prediction models stratified by deciles. The recalibrated Framingham model consistently underestimated HT incidence, especially in high-risk groups. The BP-only model overestimated risk in the lower deciles. With the exception of the highest decile, risk predicted by the KoGES model was very close to the observed risk compared with the Framingham and BP-only models (χ²=14.85, P=0.062).

Figure 2.

Observed and predicted probabilities according to the Korean Genome Epidemiology Study (KoGES) and Framingham models. P=0.062 for the observed vs. predicted risk by KoGES, P<0.001 for observed vs. recalibrated Framingham, and P<0.001 for observed vs. blood pressure-only model.

The KoGES model improved continuous NRI (0.354, 95% CI: 0.343–0.365; and 0.542, 95% CI: 0.524–0.561, respectively) and IDI (0.023, 95% CI: 0.022–0.024; and 0.041, 95% CI: 0.040–0.043, respectively) over the Framingham and BP-only models.

Discussion

In this validation study of a large, nationwide sample cohort, the HT risk score according to the KoGES model showed good discrimination and calibration for predicting the onset of HT. Additionally, discrimination was significantly better with the KoGES model than with the recalibrated Framingham or BP-only model, and the estimated risk of HT calculated by the KoGES model was close to the observed incidence. Reclassification analysis showed that the KoGES model led to a statistically significant improvement over the recalibrated Framingham and BP-only models (NRI, 0.354; 95% CI, 0.343–0.365 and 0.542; 95% CI, 0.523–0.561, respectively).

A previous study showed that using a risk score to predict the incidence of HT may not be accurate in ethnic groups in which the score has not been validated.¹³ Most risk scores used to predict HT have not been validated externally in a different data source. According to a recent review article, only the Framingham and Hopkins scores have been evaluated in populations or ethnic groups that are different from those used for initial development of the model.¹⁵ Data from the Whitehall II Study¹¹ validated the HT risk score initially developed in the Framingham Offspring Study by showing that the score had good discrimination and calibration, and that the ratio of predicted to observed risk was close to 1.00 among various subgroups. However, these 2 independent cohorts both consisted of white populations. The Multi-Ethnic Study of Atherosclerosis showed that the Framingham model significantly underestimates the risk for HT, and the recalibration is necessary to correct this difference between observed and predicted risk for HT.²² Similarly, our previous study in a Korean population demonstrated that the Framingham risk score underestimated risk for HT even though the individuals in both studies had similar risk factors at baseline.¹³ Although the discrimination was similar to that in the Framingham Offspring Study, concordance between observed and predicted risk was poor even after recalibration of the Framingham model, suggesting that risk models may not be accurate in different ethnic populations even after adjusting the model. Another study in a Chinese cohort showed that the Framingham model had good calibration but lower discrimination than risk models developed from the original cohort.¹²

Differences in ethnicities based on genetic variation, cultural or environmental factors, and socioeconomic status may influence the etiology, prevalence, progression, and outcome of various diseases.²³ Many previous studies show differences in the prevalence of CVD risk factors across ethnicities.^24–28 Several studies have shown that Asians have a higher risk of HT, CVD, and all-cause death than white Europeans with the same BMI.^29–31 In clinical practice, differences in health outcomes among ethnicities may be important for implementation of health interventions.³² Genetic variants between ethnicities may also contribute to differences in CVD-related risk factors,³³ and additional validation studies of risk score models using independent data sources will help clarify these discrepancies.

Applying the HT prediction model in the clinical practice may have some positive advantages: (1) improve adherence by high-risk individuals for intensive interventions, (2) motivate high-risk individuals to sustain their efforts in lifestyle modification, and (3) provide personalized medicine and care according to risk category.

The strengths of our study include the use of a large, nationwide sample cohort extracted by stratified random sampling, with proportional allocation among 1,476 categories stratified by sex, age, and income level. These advanced stratification methods suggest the KoGES risk score accurately identifies individuals at high risk for incident HT. This study was also the first to validate a risk score for incident HT that had been previously developed in an Asian population.

However, the study has some limitations. First, this study used a single BP reading to determine hypertensive status, which does not account for variability in BP measurements and potential bias introduced when using single measurements. Second, the KoGES model was validated using independent data from individuals of the same ethnicity and is therefore applicable only in Korean populations. Third, although the model was valid for individuals aged 40–69 years, it may not be accurate in individuals outside this age range.

In summary, this validation study suggested that the KoGES risk score for predicting the incidence of HT had significantly better discrimination, calibration, and reclassification ability than the recalibrated Framingham risk score after a 4-year follow-up period. Furthermore, the KoGES risk score met the requirements for inclusion in the guidelines, as the performance was superior to the BP-only model, and may be useful for guiding lifestyle modifications and early treatment of HT.

Acknowledgments

This work was supported by funding from the Korea National Institute of Health [4800-4845-302-210 (2011-NG63002-00)]. This study used NHIS-NCD data, compiled by the National Health Insurance Service (NHIS). The authors alone are responsible for the content and writing of the paper.

Appendix

The 4-year risks of the KoGES, recalibrated Framingham, and blood pressure-only models for incident hypertension were calculated as follows.

(1) Estimated risk calculated by the KoGES model.

where, k=ln(4)–30.3547–0.2838× age –0.2084×women–0.0642×SBP–0.2033×DBP–0.2912×smoke–0.0875×parental history–0.0665×BMI–0.0031×age×DBP

(2) Estimated risk calculated by the Framingham recalibrated model.

where, k=ln(4)–22.9495–0.1564×age–0.2029×women–0.0593×SBP–0.1285×DBP–0.1907×smoke–0.1567×parental history–0.0339×BMI–0.0016×age×DBP

(3) Estimated risk calculated by the blood pressure-only model.

where, k=ln(4)–0.0375×SBP–0.0291×DBP

References

• 1.    Lim  SS,  Vos  T,  Flaxman  AD,  Danaei  G,  Shibuya  K,  Adair-Rohani  H, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012; 380: 2224–2260.
• 2.    Chen  J. Epidemiology of hypertension and chronic kidney disease in China. Curr Opin Nephrol Hypertens 2010; 19: 278–282.
• 3.    Lewington  S,  Clarke  R,  Qizilbash  N,  Peto  R,  Collins  R, Prospective Studies C. Age-specific relevance of usual blood pressure to vascular mortality: A meta-analysis of individual data for one million adults in 61 prospective studies. Lancet 2002; 360: 1903–1913.
• 4.    Kario  K,  Ogawa  H,  Okumura  K,  Okura  T,  Saito  S,  Ueno  T, et al. SYMPLICITY HTN-Japan: First randomized controlled trial of catheter-based renal denervation in Asian patients. Circ J 2015; 79: 1222–1229.
• 5.   World health Organization. A global brief on hypertension: Silent killer, global public health crisis. Geneva: WHO, 2013.
• 6.    Julius  S,  Nesbitt  SD,  Egan  BM,  Weber  MA,  Michelson  EL,  Kaciroti  N, et al. Feasibility of treating prehypertension with an angiotensin-receptor blocker. N Engl J Med 2006; 354: 1685–1697.
• 7.    He  J,  Whelton  PK,  Appel  LJ,  Charleston  J,  Klag  MJ. Long-term effects of weight loss and dietary sodium reduction on incidence of hypertension. Hypertension 2000; 35: 544–549.
• 8.   The Trials of Hypertension Prevention Collaborative Research Group. Effects of weight loss and sodium reduction intervention on blood pressure and hypertension incidence in overweight people with high-normal blood pressure: The Trials of Hypertension Prevention, phase II. Arch Intern Med 1997; 157: 657–667.
• 9.    Spencer  RM,  Heidecker  B,  Ganz  P. Behavioral cardiovascular risk factors: Effect of physical activity and cardiorespiratory fitness on cardiovascular outcomes. Circ J 2015; 80: 34–43.
• 10.    Parikh  NI,  Pencina  MJ,  Wang  TJ,  Benjamin  EJ,  Lanier  KJ,  Levy  D, et al. A risk score for predicting near-term incidence of hypertension: The Framingham Heart Study. Ann Intern Med 2008; 148: 102–110.
• 11.    Kivimaki  M,  Batty  GD,  Singh-Manoux  A,  Ferrie  JE,  Tabak  AG,  Jokela  M, et al. Validating the Framingham Hypertension Risk Score: Results from the Whitehall II study. Hypertension 2009; 54: 496–501.
• 12.    Chien  KL,  Hsu  HC,  Su  TC,  Chang  WT,  Sung  FC,  Chen  MF, et al. Prediction models for the risk of new-onset hypertension in ethnic Chinese in Taiwan. J Hum Hypertens 2011; 25: 294–303.
• 13.    Lim  NK,  Son  KH,  Lee  KS,  Park  HY,  Cho  MC. Predicting the risk of incident hypertension in a Korean middle-aged population: Korean genome and epidemiology study. J Clin Hypertens (Greenwich) 2013; 15: 344–349.
• 14.    Otsuka  T,  Kachi  Y,  Takada  H,  Kato  K,  Kodani  E,  Ibuki  C, et al. Development of a risk prediction model for incident hypertension in a working-age Japanese male population. Hypertens Res 2015; 38: 419–425.
• 15.    Echouffo-Tcheugui  JB,  Batty  GD,  Kivimaki  M,  Kengne  AP. Risk models to predict hypertension: A systematic review. PLoS One 2013; 8: e67370, doi:10.1371/journal.pone.0067370.
• 16.    D’Agostino  RB Sr,  Grundy  S,  Sullivan  LM,  Wilson  P; CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: Results of a multiple ethnic groups investigation. JAMA 2001; 286: 180–187.
• 17.    Weiner  DE,  Tighiouart  H,  Elsayed  EF,  Griffith  JL,  Salem  DN,  Levey  AS, et al. The Framingham predictive instrument in chronic kidney disease. J Am Coll Cardiol 2007; 50: 217–224.
• 18.    Yatsuya  H,  Iso  H,  Li  Y,  Yamagishi  K,  Kokubo  Y,  Saito  I, et al. Development of a risk equation for the incidence of coronary artery disease and ischemic stroke for middle-aged Japanese: Japan Public Health Center-Based Prospective Study. Circ J 2016 April 15, doi:10.1253/circj.CJ-16-0081.
• 19.    Khang  YH,  Bahk  J,  Yi  N,  Yun  SC. Age- and cause-specific contributions to income difference in life expectancy at birth: Findings from nationally representative data on one million South Koreans. Eur J Public Health 2016; 26: 242–248.
• 20.    DeLong  ER,  DeLong  DM,  Clarke-Pearson  DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 1988; 44: 837–845.
• 21.    Hosmer  DW,  Hosmer  T,  Le Cessie  S,  Lemeshow  S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 1997; 16: 965–980.
• 22.    Muntner  P,  Woodward  M,  Mann  DM,  Shimbo  D,  Michos  ED,  Blumenthal  RS, et al. Comparison of the Framingham Heart Study hypertension model with blood pressure alone in the prediction of risk of hypertension: The Multi-Ethnic Study of Atherosclerosis. Hypertension 2010; 55: 1339–1345.
• 23.    Burchard  EG,  Ziv  E,  Coyle  N,  Gomez  SL,  Tang  H,  Karter  AJ, et al. The importance of race and ethnic background in biomedical research and clinical practice. N Engl J Med 2003; 348: 1170–1175.
• 24.    Takayama  T,  Hirai  S,  Oka  S,  Ishihara  T,  Kumazaki  S,  Mishima  H, et al. Bilateral giant bullae with rapidly increasing fluid in the right bulla following operation: Report of a case. Surg Today 1994; 24: 272–275.
• 25.    Cowie  CC,  Harris  MI,  Silverman  RE,  Johnson  EW,  Rust  KF. Effect of multiple risk factors on differences between blacks and whites in the prevalence of non-insulin-dependent diabetes mellitus in the United States. Am J Epidemiol 1993; 137: 719–732.
• 26.    Sprafka  JM,  Folsom  AR,  Burke  GL,  Edlavitch  SA. Prevalence of cardiovascular disease risk factors in blacks and whites: The Minnesota Heart Survey. Am J Public Health 1988; 78: 1546–1549.
• 27.    Teramoto  T,  Ohashi  Y,  Nakaya  N,  Yokoyama  S,  Mizuno  K,  Nakamura  H, et al. Practical risk prediction tools for coronary heart disease in mild to moderate hypercholesterolemia in Japan: originated from the MEGA study data. Circ J 2008; 72: 1569–1575.
• 28.    Ohira  T,  Iso  H. Cardiovascular disease epidemiology in Asia: An overview. Circ J 2013; 77: 1646–1652.
• 29.    Shai  I,  Jiang  R,  Manson  JE,  Stampfer  MJ,  Willett  WC,  Colditz  GA, et al. Ethnicity, obesity, and risk of type 2 diabetes in women: A 20-year follow-up study. Diabetes Care 2006; 29: 1585–1590.
• 30.    Wen  CP,  David Cheng  TY,  Tsai  SP,  Chan  HT,  Hsu  HL,  Hsu  CC, et al. Are Asians at greater mortality risks for being overweight than Caucasians?: Redefining obesity for Asians. Public Health Nutr 2009; 12: 497–506.
• 31.    Deurenberg  P,  Deurenberg-Yap  M,  Guricci  S. Asians are different from Caucasians and from each other in their body mass index/body fat per cent relationship. Obes Rev 2002; 3: 141–146.
• 32.    Schulman  KA,  Berlin  JA,  Harless  W,  Kerner  JF,  Sistrunk  S,  Gersh  BJ, et al. The effect of race and sex on physicians’ recommendations for cardiac catheterization. N Engl J Med 1999; 340: 618–626.
• 33.    Mitchell  BD,  Kammerer  CM,  Blangero  J,  Mahaney  MC,  Rainwater  DL,  Dyke  B, et al. Genetic and environmental contributions to cardiovascular risk factors in Mexican Americans: The San Antonio Family Heart Study. Circulation 1996; 94: 2159–2170.