Implementation of Voice-Controlled Remote Measurements via Metaverse using Automatic Speech Recognition

Abstract

Research on the three-dimensional virtual space “metaverse” is being actively conducted. The metaverse allows communication between distant “people” through voice and/or gestures. Applying the metaverse to distant “objects” is expected to bring industrial innovation. The industrial metaverse can be implemented in remote measurement systems. In previous research, we successfully controlled a measurement instrument using voice control via the metaverse. However, appropriate automatic speech recognition methods for metaverse measurement instruments have not been explored. In this study, we investigate the performance of representative automatic speech recognition methods that use deep neural networks, including Google Speech-to-Text and Faster-Whisper, an end-to-end generative pretrained transformer. In the case of dedicated control commands with the prefix, high performance was achieved with an average word error rate of 2%. Based on the results, we developed a measurement system and successfully demonstrated that it is possible to control measurement instruments in remote sites via the metaverse.