A Development of Wearable System for Information Providing Applications Using Scene Text Recognition

Abstract

Smartphone Apps that recognize texts captured with a camera and provide information based on the recognized texts are becoming widespread. However, they are hard to be used in situations where hands cannot be freely used, and also for handicapped people. There are smart glasses as a device suitable for the information providing system under such situations, however, there were no items that had the performance necessary for character recognition in the scene image and did not interfere with work and actions. In this paper, we developed a wearable system for information providing applications using scene text recognition. The prototype device has a camera suitable for character recognition in the scene image and a display that does not hinder the tasks and actions while wearing. In addition, it has a function to communicate with an external PC or a cloud server, and APIs for that are also developed. Furthermore, since it has no input device except for the camera, we developed a user interface suitable for such a device. We implemented a sample application that recognizes a station name on the route map and displays information on the station. We conducted evaluation experiments of the user interface and the whole system, and confirmed that it is practical for tasks that are allowed to retry.