Advanced Biomedical Engineering
Online ISSN : 2187-5219
ISSN-L : 2187-5219
Operating Room Surveillance Video Analysis for Group Activity Recognition
Koji YokoyamaGoshiro YamamotoChang LiuKazumasa KishimotoTomohiro Kuroda
Author information

2023 Volume 12 Pages 171-181


Human error during surgery can cause unexpected accidents. Elucidating the causes of human error by monitoring the behavior of the surgical team can improve safety. Conventional methods include a questionnaire that is presented to the surgical team, but this is subjective, and its accuracy depends on memory. However, pose estimation is an emergent technology that is able to capture a person's skeletal information. Objective monitoring of group behaviors using pose estimation may assist in identifying the causes of human error. In this study, we utilized the surveillance video images of the entire surgical team to verify the effectiveness of the proposed method in extracting and quantifying group behaviors in order to identify the intraoperative situation. Specifically, we attempted to extract the behaviors focusing on the actions of handing over surgical instruments and the actions that multiple members pay attention to simultaneously. In the proposed method, pose estimation is applied to surveillance camera images to model individual behavior based on the individual's position, facial orientation, body orientation, arm bending, and wrist position. The individual's behavior and ID are then combined and input into an estimation model to extract group behavior. The proposed method was applied to six actual surgical videos to estimate group behaviors. Accordingly, we were able to accurately extract behaviors involving cooperation of multiple members in addition to behaviors involving receiving and delivering, as well as behaviors warranting the attention of multiple members. Further, we successfully identified important points of occurrence of characteristic group behaviors from redundant surgical videos.

Content from these authors
© 2023 Japanese Society for Medical and Biological Engineering

Copyright: ©2023 The Author(s). This is an open access article distributed under the terms of the Creative Commons BY 4.0 International (Attribution) License (, which permits the unrestricted distribution, reproduction and use of the article provided the original source and authors are credited.
Previous article Next article