Performance Evaluation of a Method for Recognizing Utterances in Local Assembly Minutes

Abstract

We describe performance evaluation of a method for recognizing utterances in local assembly minutes. The experimental datasets were collected from local assembly minutes of four municipalities for 4 years from April 2011 to March, 2013. The four municipalities are Tokyo, Aomori, Osaka and Fukuoka. We manually annotated each sentence whether the sentence is an utterance or not. In the experiment using the data of 4 years, we conducted two experiments using the different dataset between learning data and test data. In other words, we used ”data of same municipality” and ”data of different municipalities” for the learning data and the test data. As a result, the average correct answer rate of SVM was the highest, 0.985, 0.951 respectively. In addition, we conducted an experiment for reducing learning data that the dataset consists of assembly minutes of one year. As a result, the average correct answer rate of LSTM was 0.926, which was 0.061 higher than the average correct answer rate of SVM.