Evaluation of effectiveness of majority voting method in crowdsourcing-based subtitling methods

Abstract

Crowdsourcing is an effective way to generate subtitles for videos with multiple speakers. This study proposes and evaluates a word-by-word majority voting method as a new approach to creating high-accuracy subtitles from multiple types of subtitles created by multiple workers in crowdsourcing. In the word-by-word majority voting method, the accuracies of the subtitles themselves remarkably vary between high and low, since even a slight difference in word order, for example, can significantly reduce incorrect generation. This remarkable difference shows that the proposed method has potential to generate appropriate subtitles for videos having multiple speakers.