Research and Development of an Educational Augmented Reality Auscultation Simulator

Abstract

We propose an educational augmented reality auscultation simulator, EARS, which can seamlessly perform medical interviews and auscultation on simulated patients during physical assessment practice and examinations. This paper first describes the structure of EARS, which consists of a stethoscope with a built-in camera and a chest piece with a built-in contact sensor. Next, we describe the principle of operation of EARS, which is a method of detecting a simulated patient and chest piece using deep learning, and the procedure of calculating the auscultation position based on the detected position information. The results of a questionnaire survey conducted on EARS users in a medical education setting are presented.