Scene Segmentation for Instruction Videos by Learning Similarity between Video and Sentence

Abstract

In industry such as manufacturing or restaurant, many companies use videos of experts’ work to efficiently transfer their skills to unskilled workers. However, long videos decrease the efficiency of skill transmission because it takes long time to watch the videos. Although scene retrieval in the videos increases efficiency, it requires dividing the videos by work process. Instructional videos usually contain less visual changes compared to usual, and previous scene segmentation methods that depend only on appearance changes often fail to divide instruction videos appropriately. In this study, we propose a scene segmentation method that utilizes sentence information about work processes such as instruction manuals.