Digitalomics: Towards Artificial Intelligence / Machine Learning-Based Precision Cardiovascular Medicine
Article ID: CJ-24-0865
Details
Article ID: CJ-24-0865
Recent advances in traditional “-omics” technologies have provided deeper insights into cardiovascular diseases through comprehensive molecular profiling. Accordingly, digitalomics has emerged as a novel transdisciplinary concept that integrates multimodal information with digitized physiological data, medical imaging, environmental data, electronic health records, environmental records, and biometric data from wearables. This digitalomics-driven augmented multiomics approach can provide more precise personalized health risk assessments and optimization when combined with conventional multiomics approaches. Artificial intelligence and machine learning (AI/ML) technologies, alongside statistical methods, serve as key comprehensive analytical tools in realizing this comprehensive framework. This review focuses on two promising AI/ML applications in cardiovascular medicine: digital phonocardiography (PCG) and AI text generators. Digital PCG uses AI/ML models to objectively analyze heart sounds and predict clinical parameters, potentially surpassing traditional auscultation capabilities. In addition, large language models, such as generative pretrained transformer, have demonstrated remarkable performance in assessing medical knowledge, achieving accuracy rates exceeding 80% in medical licensing examinations, although there are issues regarding knowledge accuracy and safety. Current challenges to the implementation of these technologies include maintaining up-to-date medical knowledge and ensuring consistent accuracy of outputs, but ongoing developments in fine-tuning and retrieval-augmented generation show promise in addressing these challenges. Integration of AI/ML technologies in clinical practice, guided by appropriate validation and implementation strategies, may notably advance precision cardiovascular medicine through the digitalomics framework.
In recent years, traditional “-omics” technologies have transformed our understanding of cardiovascular disease mechanisms through comprehensive molecular profiling across multiple biological layers.1 Through integration of genome, transcriptome, proteome, metabolome, and microbiome data, researchers have elucidated complex pathophysiological pathways and identified novel therapeutic targets in cardiovascular medicine.2,3 Recent advances in single-cell multiomics approaches have unveiled previously unknown cellular heterogeneity in cardiac tissues and provided unprecedented insights into disease progression mechanisms, particularly in heart failure and atherosclerosis.4,5 Furthermore, the integration of epigenomics into genomics has enhanced our understanding of gene regulation in cardiovascular diseases, revealing crucial mechanisms in cardiac remodeling and vascular dysfunction.6
In addition to these conventional multiomics approaches, digitalomics has emerged as an innovative transdisciplinary concept that integrates multimodal digital data streams.7 These data streams include digitized physiological data, medical imaging, environmental data, electronic health records (e.g., clinical information from daily charts, laboratory findings, and annual health checkups), environmental records, and biometric data from wearables, particularly mobile health (mHealth) and extended reality (XR) devices (Figure 1). The augmented multiomics approach aims to integrate these diverse digital elements with traditional multiomics approaches to provide more precise personalized health risk assessments and customized recommendations for health optimization. This digitalomics-driven approach requires incorporation of cutting-edge digital technologies, with artificial intelligence (AI) and machine learning (ML) serving as pivotal analytical tools along with conventional statistical methods (Figure 1).8 Notably, application of AI/ML to conventional multiomics datasets and/or wearable-derived biometric data can result in the development of more precise predictive models for cardiovascular outcomes and personalized therapeutic strategies, improving patient stratification and treatment selection.9,10 Of these digital technologies, this review focuses on AI/ML, which can prove invaluable in identifying novel biomarkers for early disease detection, monitoring treatment responses in various cardiovascular conditions, and achieving precision cardiovascular medicine.
Framework of digitalomics-driven augmented multiomics. AI, artificial intelligence; EHR, electronic health records; ML, machine learning; XR, cross reality.
AI can make accurate and advanced inferences from a vast amount of knowledge data.11 However, no definition exists that can unequivocally define this concept in its entirety. Currently, ML represents the forefront of AI used in healthcare.12–15 Particularly in recent times, the development of computational resources and easy access to big data have considerably advanced the use and clinical application of AI/ML, including its subfield, deep learning.12 Regarding the use of AI in medicine, the main objectives include inference (e.g., prediction or classification) and generation. In this review, two examples have been selected to attempt to apply AI/ML to the medical settings: digital phonocardiography (PCG) and AI text generators.
Ever since René Laennec’s invention of the stethoscope in 1816, cardiac auscultation has remained a cornerstone of physical examinations in clinical medicine.16 However, the auscultatory findings are highly subjective and influenced by the examiner’s expertise and experiences.17,18 Digital PCG has emerged as a solution to objectively record acoustic data and graphically represent heart sound characteristics.19–21 Furthermore, simultaneous acquisition of electrocardiogram (ECG) data enables a more precise assessment of cardiac and valvular function.22 Despite PCG’s capability to objectively capture heart sounds, many clinicians still find it challenging to interpret the complex wavelet data and numerous features necessary for the accurate identification of murmurs and cardiac pathologies.23 Therefore, other than objective recording capabilities, an automatic and reproducible PCG interpretation pipeline is essential to increase the clinical utility of this technology.
AI/ML approaches have emerged as promising solutions to these challenges, leveraging conventional heart sound features alongside PCG-derived waveform data.17 ML models can successfully identify patients with diastolic dysfunction,24 detect low left ventricular ejection fraction,25 and diagnose valvular heart diseases.26 Herein, we present an advanced PCG system incorporating an AI/ML model for prediction of B-type natriuretic peptide (BNP) concentrations and classification of valvular heart disease severity.
The digital PCG system, marketed as the “Super StethoScope” (AMI-SSS01 series) by AMI, Inc. (Kagoshima, Japan), has been developed to automatically predict BNP levels and classify valvular disease severity using auscultation sounds and ECG through a specialized AI/ML model.27 The system’s fundamental configuration integrates a bipolar ECG with 4 frequency band-classified phonocardiograms. In typical clinical practice, healthcare providers obtain waveforms from 1 to 5 chest locations, recording data for only 8 seconds from each site to evaluate features indicative of heart failure, valvular diseases, or other cardiac conditions. Figure 2A shows normal ECG/PCG waveforms, whereas Figure 2B shows waveform patterns characteristic of moderate aortic stenosis. The device maintains signal integrity across a frequency range of 20–600 Hz within 6 dB, minimizing attenuation and thereby facilitating detection of extra heart sounds, particularly the low-frequency S3 and S4 components.27
Wavelet data obtained from a digital phonocardiogram. (A) Normal control. (B) Representative case of aortic stenosis. ECG, electrocardiogram; PCG, phonocardiogram.
The wavelet analyzer, AMI’s proprietary ECG/PCG waveform analysis system, is specifically designed for heart sound analysis and visualization (Figure 3). In the frequency analysis, in addition to the raw ECG and PCG waveforms, the system uses both the short-time Fourier transform spectrograms and continuous wavelet transform scalograms. This arrangement of heart sound waveforms in the time–frequency domain enables visual identification of acoustic features specific to various cardiac and valvular conditions. Furthermore, by generating and combining these acoustic features, AMI has been developing ML models to estimate a cardiac stress index (analogous to blood BNP levels) and predict valvular disease severity. The models primarily use 1-dimensional convolutional neural network (CNN) architectures, incorporating supervised learning and transfer learning, achieving high inference accuracy (Figure 4). A secure cloud-based service has already been launched, wherein input ECG and PCG data are processed through these AI/ML models to infer cardiac load indices and valvular disease severity, with the analyzed results then being reviewed by specialist physicians before being returned as reports on the wavelet analyzer platform. Currently, regulatory approval procedures for the AI/ML diagnostic support functionality intended for direct use by clinicians as medical devices are underway. The clinical performance of this PCG has demonstrated the potential to surpass traditional auscultation skills of physicians.27 We evaluate the ability of the AI medical device to predict “future” cardiovascular events (i.e., before their onset) based solely on auscultation sounds.
Example of acoustic sound visualization for frequency analysis by a wavelet analyzer. In the frequency analysis, in addition to the raw electrocardiogram (ECG) and phonocardiogram (PCG) waveforms, the system uses short-time Fourier transform (STFT) spectrograms and continuous wavelet transform (CWT) scalograms with appropriate transformation (log-transformation or linear regression).
Prediction of the cardiac stress index (IndexCS) and severity of valvular dysfunction via digital phonocardiography and machine learning. (A) Overall prediction pipeline. (B) Prediction algorithm for the IndexCS. (C) Prediction algorithm for the aortic stenosis severity index (IndexAS). CNN, convolutional neural network; CWT, continuous wavelet transform; ECG, electrocardiogram; PCG, phonocardiogram; STFT, short-time Fourier transform; W.N.L., within normal limit.
Generative AI is a type of AI capable of creating new images, movies, text, information, or music using trained ML algorithms.28 It has become widespread because of the processing of input and output in natural language with user-friendly web applications, such as Midjourney for image generation and ChatGPT for text generation.29 Because the use of ChatGPT, a generative pretrained transformer (GPT)-based chat-style web application launched in November 2022, has become widespread, we can easily process input and output in natural language through the applications. In addition, various health services using large language models (LLM) have been emerging. However, this rapid development has occurred without sufficient time to discuss their application in medicine.30
At the core of these advances lies natural language processing (NLP), which encompasses technologies that enable computers to process, understand, and generate human language without requiring users to have specialized knowledge of programing languages such as Java or Python.31 The history of NLP dates back to 1966 with the chatbot ELIZA.32 However, a notable milestone was achieved in 2011, when IBM Watson, equipped with statistical analysis-based NLP capabilities, defeated human champions in the US quiz show “ Jeoparady! ”.33 NLP excels in various applications, including text correction, language translation, chatbot-based dialog, question-answering systems, and sentiment analysis.31
The ML algorithms historically used for NLP tasks include CNNs, which have notably increased accuracy in image analysis (including ECG interpretation), and recurrent neural networks (RNNs), which have been particularly used to increase machine translation accuracy.34 However, these neural networks struggled with long-distance dependencies (the ability to maintain contextual relationships between distant words), resulting in decreased accuracy in generating longer translations.35 They also experienced computational challenges with large input datasets because of the extensive time required for token probability calculations. The introduction of attention mechanism marked a notable breakthrough, substantially enhancing the accuracy of long-text translation.36 The transformer architecture, introduced in 2017, further advanced this technology by incorporating self-attention, multihead attention, positional encoding, residual connections, and layer normalization.37 This innovation addressed the long-distance dependency limitations of RNNs while enabling parallel processing for reduced computation time, hence establishing itself as the foundation architecture for contemporary NLP language models.37 The transformer architecture spawned several derivative language models, notably Google’s bidirectional encoder representations from transformers (BERT),38 Text-to-Text Transfer Transformer (T5),39 and OpenAI’s GPT.
GPT, developed by OpenAI, is an enhanced language model based on the transformer architecture. Although the original GPT delivered state-of-the-art performance in natural language inference, question answering, and text classification, it required task-specific fine-tuning with supervised data.40 Subsequent iterations (GPT-2 and GPT-3) eliminated this requirement.41,42 GPT-3, in particular, leveraged the scaling law43 by correlating model performance with parameter count, training data volume, and computational power, incorporating 175 billion parameters and training on approximately 1 trillion words using extensive GPU resources.41 It introduced in-context learning, allowing for task adaptation through example demonstrations without parameter updates, and formalized the concept of prompting to elicit desired responses. Despite these advances, challenges persisted, including inconsistency in long-text generation, physical law violations in responses, bias control in pretrained outputs, and management of harmful or hallucinated content.
InstructGPT (also known as GPT-3.5) was developed to address these challenges, focusing on content control issues.44 This model incorporated reinforcement learning from human feedback, using extensive human feedback for model training to generate outputs aligned with human intent and instructions. It successfully reduced harmful and hallucinated content while maintaining task performance accuracy.44 InstructGPT served as the foundation for ChatGPT’s initial release in November 2022. Subsequently, GPT-4, released in March 2023, expanded upon GPT-3.5 with an estimated 1.76 trillion parameters in its mixture-of-experts architecture.45 Currently, the model continues to evolve through iterations including GPT-4o and GPT-4o1, with ongoing performance enhancements.46
ChatGPT has exhibited remarkable capabilities in appropriately responding to questions across various specialized fields without requiring task-specific fine-tuning, prompting extensive research into whether it can surpass human intellectual capability. To address this, studies focused on comparing LLM performance with that of human physicians, specifically using actual medical licensing examinations as benchmark datasets.
In this context, the pioneering study by Kung et al. evaluated ChatGPT’s (GPT-3.5) performance on the United States Medical Licensing Examination (USMLE Step 1, Step 2CK, and Step 3).47 Kung et al. analyzed ChatGPT’s accuracy in answering 350 questions from the 2022 USMLE, excluding items containing images or tables that were unprocessable by the then-available version of ChatGPT. The question formats included free-response and multiple-choice questions. Although ChatGPT failed to achieve the passing threshold (∼60% accuracy) for Step 1, it achieved or approached passing scores for Step 2CK and Step 3.47 This achievement garnered considerable attention, because previous attempts at accurately answering medical questions typically required medicine-specific pretraining or fine-tuning. The fact that a general-purpose LLM without medical task-specific optimization could achieve such high accuracy was noteworthy.
Our research group also investigated the medical applications of GPT by evaluating ChatGPT’s (GPT-3.5 and GPT-4) performance on Japan’s National Medical Licensing Examination, analyzing answer accuracy and reasoning capabilities. For instance, when tested against Japan’s National Medical Licensing Exams, the ChatGPT-4 model achieved remarkable accuracy rates of 82.7% for essential questions and 77.2% for basic and clinical questions, surpassing the minimum passing requirements for human candidates.48 These values of performance metrics were comparable to those reported in contemporaneous studies.49,50 However, analysis of the incorrect responses revealed limitations in comprehensive medical knowledge, understanding of Japan-specific healthcare systems, and mathematical problem-solving capabilities.48 More concerning were instances wherein responses included contraindicated medical recommendations, highlighting potential clinical risks. These findings suggest that during our study, reliance on GPT-generated medical content without critical evaluation was hazardous for users who lacked sufficient medical knowledge or the ability to verify medical information.
Nevertheless, ever since the publication of our ChatGPT research, various approaches, such as fine-tuning and retrieval-augmented generation, have been developed and implemented to address these known limitations.51 Furthermore, GPT-4’s image recognition capability (GPT-4V) has demonstrated potential for application in image-based examinations and clinical diagnosis.52,53 These advances are steadily resolving the issues identified as AI text generators continue to rapidly evolve towards clinical implementation.
In this article, we introduce the promising applications of AI/ML technology in cardiovascular medicine, focusing on digital PCG and AI text generators. Such AL/ML models, including LLM, are not only capable of answering questions meant for human candidates but may also become invaluable partners in addressing unmet medical needs in healthcare.54 However, currently, challenges persist in maintaining up-to-date specialized medical knowledge and ensuring the consistent accuracy of outputs. Despite these challenges, it is expected that that these AI/ML technologies will soon demonstrate their clinical utility and safety through scientific validation, potentially becoming one of the foundation models for the proposed generalist medical AI.55,56 As mentioned at the beginning of this review, these technologies will likely play a crucial role in realizing the digitalomics concept.
In conclusion, digitalomics represents a novel transdisciplinary concept that integrates multimodal digital data with clinical, imaging, environmental, and personal health information to provide a comprehensive understanding of individual health backgrounds and risks. The incorporation of digitalomics into traditional multiomics approaches creates augmented multiomics, enabling more precise personalized health risk prediction and optimized therapeutic recommendations. The AI/ML technology, in conjunction with statistical methods, serves as a fundamental digital analytical tool in achieving personalized cardiovascular health optimization through this augmented multiomics approach. This field continues to evolve rapidly, with numerous AI/ML-powered medical services already being implemented, demonstrating steady progress in clinical integration. Very similar to the early days of the now-indispensable infrastructure such as the Internet and smartphones, we are not meant to resist this technological evolution. Instead, our role is to properly understand and embrace AI/ML as a new partner in clinical practice. We can work towards creating a better future for cardiovascular medicine by maximizing the potential of these technologies through appropriate implementation.
This review is based on the presentation delivered at the Circulation Journal (CJ) - European Heart Journal (EHJ) joint session at the 88th Annual Meeting of the Japanese Circulation Society. The authors thank Toshihisa Anzai for organizing this session. The authors also express their gratitude to all their colleagues who have contributed to their biomedical research, with special thanks to Hajime Takeuchi and Shumpei Saito at AMI, Inc. During the preparation of this manuscript, the authors used Claude 3.5 Sonnet (LLM developed by Anthropic PBC, CA, USA) for English editing. In addition, English language assistance was provided by Enago’s English Editing service.
None.
A.N. and M.T. have received commissioned research grants from AMI, Inc.
All authors conceptualized the contents. A.N. researched, wrote, and edited all content in this review. All authors critically reviewed and agreed to the final version of the manuscript. A.N. is responsible for the integrity of the work as a whole.
Not applicable (no new data were analyzed in this review).
