A study on automatic addition of sound effects based on human movements for video production assistance

Abstract

This research aims to develop an application that automatically adds specific sound effects to specific movements on video. For example, we are considering adding an SE that sounds like a bone breaking when the head is tilted. To determine how the head is tilted, the coordinates of the feature points of the face in the video are acquired for each frame. Calculate the time to apply SE using the angular velocity from the feature points. The results show that the proposed method does not cause any discomfort compared to the SE timing of currently available videos.