We aim to enhance music satisfaction through an empathetic co-listening experience with an agent that expresses exciting and positive emotions corresponding to the complexity of and changes in the rhythm. Here we focused that the co-listening experience of sharing internal time through rhythm will reproduce emotions generated by the music and the empathetic emotion with other(s) in the same space. The agent’s internal model changes arousal by deviation from expectations of rhythm and pleasure by repetitive numbers of similar rhythms. The agent represents its internal state via facial expression and internal time via clapping and nodding gestures. The results show that i) the effect of facial expressions elevated the user’s immersion, enjoyment, excitement, and empathy with the agent and that ii) the rhythmic motions elevated comfort, enjoyment, and user’s immersion into the music.

View full abstract

The declining tendency of children's throwing ability due to the decrease in opportunities to play outdoors in recent years has been recognized as a problem. In light of this situation, instruction on throwing motion was newly added to physical education classes in the 2020 Courses of Study for elementary school students. However, the teaching method is left to the discretion of each teacher, and at present, no effective teaching method has been established to improve children's throwing ability. Under such circumstances, the authors developed a system to quantify the throwing motion using a camera. However, in order to provide immediate feedback to teachers and children on the improvement of throwing motion, it is necessary to clarify the factors of throwing motion that affect the improvement of throwing ability to a distance. Therefore, in this study, we conducted a measurement experiment of softball throwing for elementary school students, analyzed the throwing motion.

View full abstract

Boccia has been incorporated into many special needs schools, but commercially available boccia systems are considered expensive, require specialized knowledge for preparation, and are time-consuming, which makes them unsuitable for daily use in special needs school classes. In this study, we proposed a boccia system that can be used as a practical teaching aid in special needs education. First, we developed two types of prototypes and conducted field studies on how boccia systems can be utilized as a teaching aid. Based on this, we developed a fully functional boccia system with easy assembly using parts easily purchased and made by 3D printers and user-friendly layout for students on wheelchairs, so that it can be used by various students as teaching aids. Then, we conducted evaluation experiments to confirm that the system can be prepared before classes. We also conducted evaluation experiments with students in a special needs school and demonstrated effectiveness of the system, in which students with various disabilities could enjoy the system.

View full abstract

With the changing circumstances of the Covid-19 Disaster, many conventional face-to-face lectures are being conducted, but on-demand streaming of lectures is also expected to continue in the future. The authors have conducted on-demand lectures in which nodding actions of student characters are introduced from the viewpoint that these actions are important, but the effect of the nodding actions themselves has not yet been clarified. In this study, we analyzed the effects of introducing the nodding actions of student characters by conducting both face-to-face lecture and on-demand streaming that were based on recorded face-to-face lecture. The results suggest that changing the actions of the characters in the class can attract students’interest and encourage them to continue watching.

View full abstract

For many years, Ekman’s research on facial expressions has been the basis of research on facial expression recognition or estimation. However, in recent years, it was reported that Japanese people do not follow Ekman’s theory, and much remains to be clarified about the relationship between emotion and facial expression. In this study, we focused on the natural expression of eye region, which is unintended expressions of emotions, and analyzed the characteristics of it in comparison with facial features expressed in spontaneous emotional arousal situations. As a result, we revealed that the size of the eyes differed for each emotion and that the expression of the eye region varies greatly among individuals. In addition, we suggested that the mouth may be useful for pleasantness estimation by machine learning.

View full abstract

In modern socio-technical systems, such as aviation systems, the performance adjustment or resilience of operator personnel plays a key role in maintaining stable operations in the face of internal and external disturbances. This study aims to extract and classify competencies for resilient performance of pilots who contribute to safe aircraft operations. An interview survey was conducted with nine airline pilots, during which they were asked to describe their thoughts, feelings, and actions in response to risky situations they encountered while on duty. The narrative data obtained from the interviews was analyzed using a Modified-Grounded Theory Approach (M-GTA). The analysis revealed eight competencies for resilient performance of flight crews, aligning with the resilience potentials proposed by Hollnagel, as well as four complementary competencies (mindset, mental stability, motivation, and psychological safety) that can form the basis of resilient performance. Based on these results, a practical definition of resilience potentials for flight crews was proposed. The validity of the competencies for resilient performance and the practical definition of resilience potentials were confirmed through member checking by the nine participant pilots. These findings can be useful for designing training programs and learning environments to enhance the resilience potential of pilots.

View full abstract

This study aims to assess the usability of virtual reality (VR) evaluations for patients with unilateral spatial neglect (USN) and evaluates their visuospatial cognitive function in three dimensions. We developed a method using VR evaluation to analyze exploratory behavior in space. The evaluation tasks involved line bisection and search tasks, allowing analysis of task completion status, eye and head movements, and task completion time. To determine the reliability and usefulness of the ratings, an experiment was conducted involving 10 left USN group members and 56 non-clinical group members. The results demonstrated limited reliability among VR ratings, with observed correlations between VR and desk-based evaluations in some tasks. Notably, VR evaluation identified symptomatic cases that were undetected by desk evaluation. Overall, this study highlights the different aspects assessed by VR compared to desk-based evaluation and suggests new possibilities for evaluating visuospatial cognitive function in USN.

View full abstract

Dark patterns are designs that intentionally lead website users toward choices that are detrimental to their best interests. In recent years, the use of dark patterns on websites has grown, becoming a global issue. While typical dark patterns on websites are well-known, those associated with IoT devices remain less understood. In this study, we conducted two workshops to explore dark patterns in IoT products and identify associated risks. In the first workshop, we created a matrix to categorize existing dark patterns in IoT devices and conceptualize new approaches. In the second workshop, we developed a customer journey map to scrutinize dark patterns from the customer's point of view. Subsequently, we evaluated the effectiveness and limitations of our research methodology for identifying these risks. Our findings suggest that using a customer journey map provides several advantages over traditional methods based on existing classifications: it allows for contextual understanding of the product or service, enables a comprehensive examination of the entire process, and encourages consideration from the user's perspective.

View full abstract

Humans observe the behavior of others and exhibit conforming behavior according to the situation. In this study, we propose a method to change an individual’s walking behavior and attention by changing the number of multiple virtual agents in action. We verified whether a particular pedestrian’s walking behavior and attention to a poster on the wall changed according to the type and proportion of actions of multiple agents presented to the pedestrian. The results showed that the pedestrian’s walking speed and attention to the wall advertisement changed according to the agents’walking behavior, such as deceleration and stopping.

View full abstract

In this paper, we explore how humanists and technologists can work together in the field of HCI (human-computer interaction) at a corporate laboratory. We analyze six research cases at Xerox PARC to identify key insights. Our findings reveal three main perspectives: (1) It was found that the core responsibility for setting the research problem space has shifted over time from technologist-driven to humanity-driven research. Moreover, during the course of technologist-driven research, a conceptual shift occurred by leveraging the results of humanity-driven research. (2) Some projects led by the technologist side involve a back-and-forth transition between technologist-driven research and humanity-driven research. (3) As collaboration between the two sides becomes more process-oriented, the role of humanists expands to include setting the project’s problem space and goals. Finally, we provide insights into the future prospects for collaboration between humanists and technologists in corporate research laboratories based on our discussion of these findings.

View full abstract