FAST: Fast Annotation tool for SmarT devices

Abstract

In the evaluation of the generated results of NLG systems or the annotation of application tasks such as sentiment analysis, it is important to work with a wide range of annotators with the same attributes as in real-world applications. While such application cases often use crowdsourcing mechanisms to gather a variety of annotators, most real-world users use mobile devices such as smartphones. In this paper, we propose "FAST", an annotation tool for application tasks that focuses on the UX of mobile devices, which has not been focused on so far. In our experiments, we conducted annotations by several annotators and evaluated metrics such as speed, quality, and ease of use from the tool's logs and user questionnaires. As a result, our system can annotate faster than existing methods in a specific task while maintaining quality.