Study of a Hybrid Educational Method Using Virtual Patient Simulation and Physical Examination During the COVID-19 Pandemic

Abstract

Previous studies have shown the pedagogical benefits of using virtual patient simulation (VPS), but there are no studies using simulators and VPS together, making learner satisfaction and learning effectiveness unknown. This study investigated the satisfaction level of the educational program using simulators and VPS. From April to November 2021, 60 fifth-year medical students participated in a small-group program. Cardiac auscultation training was done for 80 min. Thereafter, VPS was administered for 70 min. A questionnaire was distributed to evaluate the overall educational program, physical examinations, and VPS program. Ninety-eight percent of the students reported participation in group discussions, and 97% reported participation levels consistent with those in medical school over five years. The physical examination rating was the median (interquartile range): 1(1-1) and 1(1-2) for the VPS. Therefore, education combining a simulator and VPS can facilitate the process from examination to diagnosis, and afford a high satisfaction level.

INTRODUCTION

In recent years, instruction using simulators has become popular in medical education in Japan. It has been reported in many studies that the learning effect of simulation education is significant [1-4]. Recently, the effectiveness of VPS, which can be performed on a personal computer, has been reported [5-8]. VPS is an educational platform that uses artificial intelligence to improve critical thinking and clinical decision-making. VPS allows users to gain knowledge through clinical reasoning by interacting with virtual patients and examining various test results. A virtual patient exhibits a dynamic pathophysiological response to user decisions [9-10]. Previous studies have shown the pedagogical benefits of using VPS, but there are no reports of studies using simulators and VPS together, therefore limiting our awareness of learner satisfaction and learning effectiveness. In this study, we investigated the satisfaction level of the educational program using simulators and VPS.

MATERIALS AND METHODS

Study Design

This study is a backward-looking observational study.

Subjects

The subjects of this study were 60 fifth-year medical students undergoing clinical training in cardiology, who received face-to-face instructions for seven months from April to November 2021.

Instructional Strategies

The duration of the educational program was 2.5 hours per session. We taught physical examination using a simulator for 80 min, followed by an ST-segment elevation myocardial infarction (STEMI) using VPS for 70 min.

Learning Tools

The cardiology patient simulator (ICHIRO™ KYOTO KAGAKU, Japan) contained 88 cases: 12 cases of normal heart sounds, 14 cases of heart disease simulations, 10 cases of arrhythmia simulations, and 52 cases of electrocardiogram (ECG) arrhythmia simulations. The sounds were recorded by real people and reproduced using a high-quality sound system [11]. We set up a VPS (Body Interact™ Coimbra, Portugal) that could be completed via computer operation and output to a large liquid crystal display so that everyone could view it and respond to the cases (Fig. 1).

Fig. 1.

Training with simulators and VPS

Cardiac Auscultation Training

Heart sound auscultation consisted of five programs. The programs consisted of identification of sounds I and II, heart sounds in arrhythmia, sounds III and IV, heart murmurs, and Levine classification. Heart murmurs related to aortic stenosis (AS), aortic regurgitation (AR), mitral regurgitation (MR), and tricuspid regurgitation (TR) were chosen. In the case of MR, to identify coarse crackles, a patient was selected, and in the case of TR, the patient was given instructions on the enhancement of the heart murmur during inspiration (Rivero-Carvallo sign) and V-wave amplification. Finally, we examined how to palpate and classify thrills in response to Levine grades of cardiac murmur level VI cases.

Case Management of STEMI

First, the students were introduced to the operation of the VPS and the learning objectives. The simulation time for the case response was set at 15 min and conducted through group discussion. The instructor did not provide any advice or intervention during the case response, but provided 45 min of feedback after the case response was completed. They discussed the conversation with the patient, performed a physical examination, and interpreted the findings. Subsequently, they reflected on the ordered tests and medical interventions. Finally, feedback was provided on the appropriate responses with a case commentary.

Questionnaire Survey

A post-training questionnaire was administered regarding the overall evaluation and physical examination of the educational program and program content of the VPS (Table 1). The questionnaire survey was based on a 5-point Likert scale with the ranges: 1 = strongly agree, 2 = agree, 3 = undecided, 4 = disagree, and 5 = strongly disagree.

Table 1. Questionnaire survey contents

Overall program evaluation 　Q1 I am satisfied with the program that combines physical exams and VPS. 　Q2 I was able to participate in the discussion. 　Q3 I think the content of the study is suitable for my level.

Virtual patient simulation 　Q4 Group casework is more effective than individual studies. 　Q5 I could understand the need for a medical interview. 　Q6 I could understand the need for a 12-lead ECG. 　Q7 I could understand the importance of time in STEMI cases. 　Q8 I was interested in STEMI cases.

Heart sound auscultation 　Q9 I could understand Sound 1/Sound 2. 　Q10 I could understand Sound 3/Sound 4. 　Q11 I could understand Systolic murmur/Diastolic murmur. 　Q12 I could understand the Rivero-Carvallo sign. 　Q13 I could understand Levine grades of cardiac murmur.

Statistical Analysis

Performance evaluation of patient care and questionnaire survey variables were expressed as medians (interquartile range). Data were analyzed using the Shapiro-Wilk test to confirm the normality of the distribution. All statistical analyses were performed using version 16 of the JMP (SAS Institute, Cary, NC, USA).

Ethics

The participants were informed of the purpose, methods, and ethical considerations of the study. The questionnaire survey was anonymized to ensure participant confidentiality. The participants were informed that the submission of the questionnaire survey was voluntary, and that whether it was submitted was not relevant to the evaluation. This study did not address individual students’ grades or the medical information held by hospitals. We explained that the results obtained in this study would be published in medical conferences and journals.

RESULTS

Results of the Questionnaire Survey

Three participants submitted blank questionnaires, while two participants did not submit. We analyzed the results of the questionnaire survey of 55 people who consented to participate in the study. In the overall program evaluation, all students were satisfied with the hybrid physical examination and VPS education. Ninety-eight percent of the respondents participated in discussions with group members and 2% were undecided. Ninety-seven percent reported that the learning content was appropriate for their level, while 3% reported that they were undecided (Fig. 2). The median scores were 1(1-1) in Q4, Q5, Q6, and Q8 for VPS and median 1(1-2) in Q7 (Fig. 3). In the open-ended comments on the VPS, 73% of the students described the learning effect of working in a group: “There is a limit to individual knowledge and judgment” and “I was able to treat patients through discussion with group members.” Three percent of the students described the limitations of the VPS. They stated that “it is difficult to understand the rationale behind the words and actions that are considered correct from the VPS alone” and “without feedback from the instructor, it is difficult to interpret the findings”. The median scores for heart sound auscultation were 1(1-1) for Q9 and Q11 and 1(1-2) for Q10, Q12, and Q13 (Fig. 3).

Fig. 2.

Overall evaluation of the hybrid education program

Fig. 3.

Learning assessment by VPS and Heart sound auscultation

Performance Evaluation of VPS

The median score for each group on the VPS was 68.5 (56.3-77.8). Students tended to order clinical examinations without completing medical interviews and physical examinations. Major clinical examinations included 12-lead ECG, arterial blood gas, biochemistry, cardiac biomarkers, lipid profile, and chest radiography.

DISCUSSION

This study investigated the students’ satisfaction with an educational program that combined a simulator-based physical examination and VPS. The measurement of learner satisfaction is the first step in Kirkpatrick’s evaluation and is an important evaluation item in human resource development. We found that learners were satisfied with education using both the simulator and VPS. Medical students are unable to visualize how the examination techniques they are learning are useful in clinical practice. In this study, we provided them with the experience of practicing the examination techniques they learned on the simulator. It is thought that the experience of interpreting the findings and reflecting on the thought process contributed to the improvement of learners’ satisfaction. The fact that appropriate feedback was given and that it was appropriate to the learner’s level was also shown to be a factor contributing to satisfaction. Many learners appreciated the VPS learning style in the groups. Previous studies have shown the usefulness of responding to cases with the help of others, and this study is consistent with these studies [12]. Owing to the COVID-19 pandemic, medical students lack adequate discussion experience. They also feel the limitations of non-face-to-face individual learning. Therefore, we believe that the learning style of discussion with group members would meet the needs of medical students. Cook et al. proposed the use of VPS as a new possibility for simulation education and demonstrated its usefulness using a meta-analysis [13]. However, other studies have shown that feedback within the VPS alone is insufficient to achieve positive learning effects [6]. Based on these results, instructor feedback is important, even when VPS is used for self-learning. Feedback through interactive discussion is expected to improve the learning effect. If the discussion is difficult, we think it is useful to provide learning aids and feedback materials prepared in advance.

The highest median rating for VPS was the concept of time in the STEMI response. STEMI requires a rapid response to reduce the total ischemic time of reperfusion [14]. Ninety-eight percent of the students said that they understood the importance of time. However, 2% of the students were undecided. The guidelines recommend taking a brief history and checking vital signs, performing continuous ECG monitoring, 12-induction ECG, and clinical examination within 10 min of a patient’s arrival [15-16]. In this training, we did not give instruction on how to apply the monitoring ECG or 12-induction ECG electrodes. Therefore, we believe that it was difficult to recall these actions within 10 min. When training medical students, it is important to carefully instruct them on thinking and examination techniques while demonstrating the behaviors required in clinical practice. In recent years, it has been pointed out that there is a tendency for physicians to order tests without conducting a physical examination first [17]. There should be a specific reason for ordering the tests. This study suggests that this tendency is present also among medical students. The importance of medical interviews and physical examinations in clinical clerkships must be conveyed. We need to conduct history-taking and physical examination to gather information that is useful for the differential diagnosis of RULE-IN or RULE-OUT.

The model core curriculum for medical education in Japan sets the goal of deriving diseases from pathological conditions and making differential diagnoses. In clinical clerkship, it is important to not only consider the basic technique but also the pathology. Therefore, we explained the relationship between MR and coarse crackles and the relationship between TR and the Rivero-Carvallo sign in the physical examination using the simulator. The Levine grade of cardiac murmur had the highest median value among the questions regarding physical examination. This was because of the difficulty in palpating the thrill. Palpation is a complex task that requires the right combination of knowledge, skills, and attitudes, thus making it a real challenge to teach [18]. However, improvements can be expected by introducing teaching methods such as the seven-step palpation method when using the simulator [18].

In this study, we combined physical examination with a simulator and VPS to enable students’ clinical reasoning skills. We found that combining the advantages of each educational tool enabled clinically relevant training and high satisfaction among medical students.

Our study has two main limitations. First, although we assessed satisfaction with the educational program, we cannot establish whether our method is superior to other educational methods. In the future, it will be necessary to compare the results with those of conventional educational methods and to measure the learning effects using multiple-choice questions. Second, physical examination using a simulator is limited to cardiovascular examination. This training program was provided during cardiology rotation; therefore, it was not possible to train a wide range of physical examinations. This may have affected the performance of VPS. However, at our university, each department actively provides simulation education, and it is possible to learn across the board during the two-year clinical clerkship.

CONCLUSIONS

In this study, we found that education combining physical examination training using a simulator and clinical reasoning using VPS can result in medical students’ high level of satisfaction.

Disclosure Statement

All authors have nothing to disclose related to the content of this study.

REFERENCES

• 1)   Kim  J,  Park  J-H,  Shin  S. Effectiveness of simulation-based nursing education depending on fidelity: A meta-analysis. BMC Med Educ 2016;16:152. doi: 10.1186/s12909-016-0672-7.
• 2)   Kern  DH,  Mainous III  AG,  Carey  M,  Beddingfield  A. Simulation-based teaching to improve cardiovascular exam skills performance among third-year medical students. Teach Learn Med 2011;23:15-20. doi: 10.1080/10401334.2011.536753.
• 3)   Swamy  M,  Bloomfield  TC,  Thomas  RH,  Singh  H,  Searle  RF. Role of SimMan in teaching clinical skills to preclinical medical students. BMC Med Educ 2013;13:20. doi: 10.1186/1472-6920-13-20.
• 4)   Abre  JM,  Guimarães  B,  Castelo-Branco  M. The role of virtual interactive simulators in medical education: Exploring their integration as an assessment methodology in clinical years. Educ Med 2021;22:325-9. doi: 10.1016/j.edumed.2021.06.011.
• 5)   Padilha  JM,  Ribeiro  A,  Rosa  J,  Marques  D,  Machado  PP. Clinical virtual simulation as a lifelong learning strategy: nurse’s verdict. Clin Simul Nurs 2020;47:1-5. doi: 10.1016/j.ecns.2020.06.012
• 6)   Watari  T,  Tokuda  Y,  Owada  M,  Onigata  K. The utility of virtual patient simulations for clinical reasoning education. Int J Environ Res Public Health 2020;17:5325. doi: 10.3390/ijerph17155325.
• 7)   De Ponti  R,  Marazzato  J,  Maresca  AM,  Rovera  F,  Carcano  G,  Ferrario  MM. Pre-graduation medical training including virtual reality during the COVID-19 pandemic: A report on students’ perception. BMC Med Educ 2020;20:332. doi: 10.1186/s12909-020-02245-8.
• 8)   Pottle  J. Virtual reality and the transformation of medical education. Future Healthc J 2019;6:181-5. doi: 10.7861/fhj.2019-0036.
• 9)  Body Interact HP. Body Interact. https://web.bodyinteract.com/WebGL/
• 10)   Padilha  JM,  Machado  PP,  Ribeiro  A,  Ramos  J,  Costa  P. Clinical virtual simulation in nursing education: A randomized controlled trial. J Med Internet Res 2019;21:e11529. doi: 10.2196/11529.
• 11)   Kyoto  Kagaku. Cardiology Patient Simulator “K” ver.2. http://www.kyotokagaku.com/products_data/mw10_catalog_en.pdf
• 12)   Ball  R,  Sherwood  E,  Ryznar  R,  LaPorta  A,  Choudhury  Q. Evaluation of the body interact virtual patient simulator device in training medical students. Rocky Vista University College of Osteopathic Medicine. https://bodyinteract.com/docs/Evaluation_of_Body_Interact_Virtual_Patient_Simulator_in_Training_Medical_Students.pdf
• 13)   Cook  DA,  Erwin  PJ,  Triola  MM. Computerized virtual patients in health professions education: A systematic review and meta-analysis. Acad Med 2010;85:1589-602. doi: 10.1097/ACM.0b013e3181edfe13.
• 14)   Kimura  K,  Kimura  T,  Ishihara  T, et al. JCS 2018 Guideline on Diagnosis and Treatment of Acute Coronary Syndrome. Circ J 2019;83:1085-196. doi: 10.1253/circj.CJ-19-0133.
• 15)   Diercks  DB,  Peacock  WF,  Hiestand  BC, et al. Frequency and consequences of recording an electrocardiogram >10 minutes after arrival in an emergency room in non-ST-segment elevation acute coronary syndromes (from the CRUSADE Initiative). Am J Cardiol 2006;97:437-42. doi: 10.1016/j.amjcard.2005.09.073.
• 16)   Rokos  IC,  French  WJ,  Koenig  WJ, et al. Integration of Pre-Hospital Electrocardiograms and ST-Elevation Myocardial Infarction Receiving Center (SRC) Networks: Impact on Door-to-Balloon Times Across 10 Independent Regions. JACC Cardiovasc Interv 2009;2:339-46. doi: 10.1016/j.jcin.2008.11.013.
• 17)   Takashina  T. After 200 years, time to watch the step of a stethoscope. Journal of the Japanese Society of Internal Medicine 2016;105:861-5. doi: 10.2169/naika.105.861. (in Japanese)
• 18)   Aubina  A,  Gagnona  K,  Morin  C. The seven-step palpation method: A proposal to improve palpation skills. Int J Osteopath Med 2014;17:66-72. doi: 10.1016/j.ijosm.2013.02.001.