Abstract
A novel task for evaluating question answering technologies is proposed.This task assumes interactive use of question answering systems and evaluates among other things, the abilities needed under such circumstances, i.e.proper interpretation of questions under a given dialogue context;in other words, context processing abilities such as anaphora resolution and ellipses handling.This paper shows the design of the task and its empirical background.The task proposed is not only novel as an evaluation of the handling of information access dialogues, but also includes several valuable ideas such as a measuring metric in order to obtain intuitive evaluation of the answers to list-type questions and reference test sets for obtaining information on context processing ability in isolation.
