Driver Action Recognition is a key component in driver monitoring systems, which is helpful for the safety management of commercial vehicles. Compared with traditional human action recognition tasks, driver action recognition is required to be fast and accurate on embedded systems. We propose a fast and accurate driver action recognition method that is composed of CNN based driver pose estimation and RNN based driver action recognition. We train our network model with multi-task learning includes localizing and detecting each body part of the driver, classifying state of each body part, and recognizing driver action at once. Our multi-task learning for the proposed model achieves a significant improvement compared to state-of-the-art human action recognition methods with limited computational resources. We also perform ablation study of our methods which composed of the driver pose localization, detection, and classification.

How can we make it possible for humans to participate in the robot’s speech to aim for a co-constructed conversation? In this paper, we investigated the effects and factors of dialogue design, focusing on “incompleteness.” We examined the people’s attitudes toward participation in multi-party conversations using “human-robot assisted story-telling” interactions. The results showed that the utterance strategy of lacking words reduced the passive participation attitude when the talker robot speak to humans directly. If we want to increase people’s participation attitude in a conversation, avoiding conveying much information and using “incompleteness” is an effective way to do so. However, the results also confirmed that the incomplete utterance was not satisfied to improve people’s co-telling attitude yet. The robots in this study were unable to accept the variety of ways in which people speak. To achieve the co-constructed conversation, discussed how robots could install a variety of actions based on other multi-party conversation studies. Therefore, we also investigated the limitation of multi-party participation and the characteristics of human speeches. For people and systems to have a co-constructed conversation rather than as information transfer, we believe that the design of dialogue needs to change. For this reason, we reported one of the effects of “incompleteness” conversation design here.

Dialogue systems, which give users quick and easy access to required information interactively, have been widely used in various fields. Dialogue systems equipped with interfaces (e.g., humanoid robots and anthropomorphic agents) have been developed in order to enhance familiarity and dialogue continuity. Related studies, in which interactive agents generate humor expressions, have also been reported. Humor is indispensable for the formation of friendly relationships between people and systems, and humor expressions can be applied in situations that generate familiar responses and provide fun to users. In this study, in order to evoke humor through dialogue, a method to generate humorous expression by asking again due to pseudo mishearing of a part of users’ queries based on examples is proposed. Specifically, a conversion candidate dictionary for humor expressions, based on Wikipedia of Japanese edition and a classification vocabulary table in which words are classified semantically, is created by word completion using distributed representation. In addition, a word conversion method is designed by approximately 1,000 mishearing survey from Twitter, and the function based on the proposed method is implemented in a dialogue system introduced into a university as a model case. In the results of the comparative evaluation with other methods quantitatively, the proposed methods gave users the most humor by converting singular and multiple words. Thus, the effectiveness of the proposed method was clarified.

In research on semi-autonomous telepresence robots, a problem in which remote operators become frustrated with autonomous operations that do not match their intention has been reported. Previous researches proposed the methods for automatically switching between remote and autonomous operations in order to avoid the problem. However, those methods switch based on the sign depended on a task. In this paper, through the use of a general purpose arbitration model, called the accumulator based arbitration model (ABAM), we propose a semi-autonomous telepresence robot architecture for adaptively switching between remote and autonomous operations, named “One Minder”. ABAM inhibit autonomous operation according to remote operation so that One Minder can switch without the sign depended on a task. We incorporated One Minder into a semi-autonomous telepresence system autonomizing contingent behaviors, and conducted experiments to verify its utility from two aspects of remote operator and local user who talks with remote operator via a telepresence robot. The experiment results indicate that One Minder can adaptively switch between remote and autonomous operations without manual switching. In addition, One Minder was able to reduce the operational load and frustration given to a remote operator by allowing the arbitration to properly output an autonomous operation. Moreover, the arbitration of One Minder suppressed a negative influence of collision on a local user.