Feasibility Study on Extracting Peak Moments in Video by Spectrogram Analysis with Deep Learning

Abstract

Video consumption using smartphones has been increasing along with the growing popularity of video hosting service. On the other hand, due to the vast number of available videos, it is difficult to watch all of them. People are increasingly viewing videos selectively at the peak moments in the videos. Therefore, there is a growing need to extract the peak moments automatically. Existing studies on extracting the peak moments from the video include those focusing on video features using image processing, those focusing on volume and pitch using audio processing, and those analyzing data other than video, such as comments. This study attempts a new approach to extracting the peak moments by focusing on spectrograms representing audio features and applying image processing to them. The evaluation experiments suggest the possibility of automatic extraction of the peak moments in sports videos.