Journal of Natural Language Processing
Online ISSN : 2185-8314
Print ISSN : 1340-7619
Evaluation Framework Design of Spoken Term Detection Study at the NTCIR-9 IR for Spoken Documents Task
Hiromitsu NishizakiTomoyosi AkibaKiyoaki AikawaTatsuya KawaharaTomoko Matsui
Author information

2012 Volume 19 Issue 4 Pages 329-350


This paper describes a design of spoken term detection (STD) studies and their evaluating framework at the STD sub-task of the NTCIR-9 IR for Spoken Documents (SpokenDoc) task. STD is the one of information access technologies for spoken documents. The goal of the STD sub-task is to rapidly detect presence of a given query term, consisting of word or a few word sequences spoken, from the spoken documents included in the Corpus of Spontaneous Japanese. To successfully complete the sub-task, we considered the design of the sub-task and the evaluation methods, and arranged the task schedule. Finally, seven teams participated in the STD sub-task and submitted 18 STD results. This paper explains the STD sub-task details we conducted, the data used in the sub-task, how to make transcriptions by speech recognition for data distribution, the evaluation measurement, introduction of the participants’ techniques, and the evaluation results of the task participants.

Information related to the author
© 2012 The Association for Natural Language Processing
Previous article