Journal of Information Processing
Online ISSN : 1882-6652
ISSN-L : 1882-6652
 
Image-to-Text Translation for Interactive Image Recognition: A Comparative User Study with Non-expert Users
Wataru KawabeYusuke Sugano
著者情報
ジャーナル フリー

2024 年 32 巻 p. 358-368

詳細
抄録

Interactive machine learning (IML) allows users to build their custom machine learning models without expert knowledge. While most existing IML systems are designed with classification algorithms, they sometimes oversimplify the capabilities of machine learning algorithms and restrict the user's task definition. On the other hand, as recent large-scale language models have shown, natural language representation has the potential to enable more flexible and generic task descriptions. Models that take images as input and output text have the potential to represent a variety of tasks by providing appropriate text labels for training. However, the effect of introducing text labels to IML system design has never been investigated. In this work, we aim to investigate the difference between image-to-text translation and image classification for IML systems. Using our prototype systems, we conducted a comparative user study with non-expert users, where participants solved various tasks. Our results demonstrate the underlying difficulty for users in properly defining image recognition tasks while highlighting the potential and challenges of interactive image-to-text translation systems.

著者関連情報
© 2024 by the Information Processing Society of Japan
前の記事 次の記事
feedback
Top