mimi and THINKET: Machine Learning Models and Wearable Devices for Spoken Dialogue Systems

Abstract

Fairy Devices is engaged in research of spoken language processing to realize smooth communications between humans and machines. Our vision is to create a device that makes people feel as if a fairy is inside. On the basis of our research, we offer a technology stack for implementing spoken dialogue systems, from multimodal sensing devices to software on edge devices to web APIs. In this demonstration, we introduce our recent research activities, including the development of turn-taking, emotion, and attitude recognition models using paralinguistic information, wearable devices equipped with cameras and microphones, and their applications.