Recent advances in the field of human cognition development have led to increased interest in the study of cross-modal correspondence. In the case of the correspondence between sound and shape, the correspondence between simple sound (e.g., pitch, timbre) and shape angularity of has been studied while there has been little research on the other features. Previous studies have shown that the emotion mediates cross-modal correspondences. Therefore, this study focuses on the complexity of tone sequences and shapes, which is said to have more influence on emotion than simple sound or shape angularity, and aims to examine the correspondences mediated by their complexity. First, indices for shape complexity (curvature entropy) and tone sequence complexity (information entropy) are selected. Then, an experiment was conducted to investigate the correspondence between shapes and tone sequences. As a result, the correspondence between the complexity of shapes and tone sequences was found.

We investigated the effects of stroking stimuli near the external auditory meatus on the emotions aroused by the experience of listening to sounds. Previous studies demonstrated that vibratory stimuli to the upper body manipulate emotions while watching videos or listening to sounds; however, none examined the effects of stroking stimuli near the external auditory meatus through which the vagus nerve extends. This study investigated the effects of stroking stimuli to the outer ear on emotions for each of six emotionally evocative sounds that arouse pleasant or unpleasant feelings. They comprised the sounds of rain, shampoo, insects’ buzz, and horrifying winds. When the sound stimuli were presented with stroking stimuli at the ear, participants’ emotions mostly changed to be more pleasant and sleepy, that is less arousing. These findings are expected to help in the development of haptic interfaces for more emotionally appealing content.

Face tracking is a necessary technique for facilitating nonverbal communication between avatars in the metaverse space. In this study, we investigated a dynamic measurement approach for real-time representation of open/closed eye. The conventional method of extracting open/closed eye based on the aspect ratio of the eye has been the mainstream method, but it encounters issues with false detection when the neck is tilted or when the user wears glasses during live transmission. To address these limitations, we developed detection method based on the ratio of the distance between the outer and inner eyelid, complemented by a calibration technique. The detection accuracy was verified by comparing the results with camera images. The open/close eye extraction system used a web camera and Google’s MediaPipe. The results demonstrated the accuracy of open/closed eye detection was 78.68%, even when the head was in motion, and 72.96% even when the participant was wearing eyeglasses.

In this paper, we discuss how to incorporate the analytic hierarchy process (AHP) into conjoint analysis (CA). When conducting a CA, analysts must make policy alternatives of attributes and their levels. The more the number of attributes and its level, the greater is the number of alternatives, which can increase the burden of addressing a questionnaire. The characteristics of AHP can be useful in controlling this problem. This paper aims at deriving the overall evaluation value of AHP from the CA questionnaire. This will help us propose a new perspective of using AHP in the procedure of CA.

It is useful to extract the time factor of gait from captured gait videos. However, this requires frame-by-frame counting, which is expensive in terms of manpower and time. In this study, we developed discrimination models to discriminate between the stance phase and the swing phase of a walker from a walking video. Using MediaPipe, a marker-less motion capture system that can be used with a single camera, we discriminated the stance phase and swing phases of a walker from the angular changes of the waist, knee, and ankle on each side of the walker in each frame. The results showed that the right leg and left leg were discriminated with 95.1% and 95.0% accuracy, respectively. The gait cycle was calculated from the discrimination results, and the average deviation was only 7.4% for the right leg and 4.2% for the left leg.

We have been developing a cognitive, brain-machine interface (BMI)-based, training system called the “Neurotrainer.” This system was initially designed to detect averaged event-related potentials (ERPs), which reflect momentary heightened attention, allowing these individuals to participate in cognitive training without the need for hands-on interaction. In this study, we expanded our method to decode single-trial ERPs during a racing game, with the expectation that neurofeedback would accelerate the training process. We assessed the performance of the prototype system with healthy volunteers. The decoding accuracy of the target character was approximately 54% for a single trial and 83% for five trials (chance level = 12.5%). Moreover, ERP responses were stronger in the feedback condition than in the no-feedback condition. These results suggest that the BMI could be an effective tool for cognitive training, as real-time neurofeedback influences the brain activation of the players.

Psychophysical and affective experiences while watching audiovisual content can be antecedents to determine the overall evaluation of viewing experiences. However, the layered relationships among these experiences have been hardly discussed. We investigated these experiences and their semantic structure while watching hug scenes. We selected 38 adjectives describing hugs and classified them into three layers: psychophysical, affective, and overall evaluation. Participants scored each of the 24 videos containing hugging scenes using these adjectives. The structure among the three layers was computed: the four psychophysical factors affected the four affective factors, which in turn affected the two overall evaluation factors, i.e., joyful and reassuring. The model was confirmed to have statistical validity by structural equation modeling and semantic validity by experiments using dummy links. The results will lead to the determination of measures to enhance affective experiences when viewing videos, and the formulation of criteria for measuring and evaluating affective experiences.

Mouthwashes provide an important means for improving oral hygiene. This study investigated the psychophysiological effects of mouthwashes with different binders during the recovery after a short-term stressor. In a within-subject experimental design, 21 healthy adults used four mouthwashes; MW A (no binder), MW B (Sodium alginate 0.16%), MW C (Sodium alginate 0.32%), and MW D (Xanthan gum 0.08%), after a 20-minute calculation task. Subjective evaluations and physiological responses including skin conductance level and electrocardiogram were recorded throughout the experiment. MW C exhibited an increased high-frequency (HF) component of heart rate variability (HRV) during recovery, and a significant difference in “strength of thickness” compared to other mouthwashes. MW D showed a greater decrease in heart rate and increased HF component of HRV compared to MW B. The findings suggest that mouthwashes with higher thickness levels effectively promote recovery from the physiological stress response, and are potentially suitable for relaxation purposes.

To estimate the garment impression, we verified three regression models using design parameters, geometrical and texture features of images, and convolutional features of images. Adopting three-dimensional apparel simulation, we generated 375 images of a men's outdoor jacket by changing design parameters, namely the length, waist, hem circumference, and sleeve circumference. Nine participants evaluated the cool-uncool (kakkoī-kakkowarui in Japanese) impression of the garment images using a semantic differential method. We used the measured widths, angles, and G-type Fourier descriptors of the jacket images as geometrical features. For texture features, the fractal dimension and Haralick texture features (Asm, Entropy, Contrast, Correlation) of the images were used. Convolutional features were obtained from the images using a convolutional neural network. By using the design parameters and the obtained features, we estimated the image impression using three regression models: multiple linear regression, neural network, and light gradient boosting machine (LightGBM) models. As a result, the LightGBM with the design parameters had the highest mean correlation coefficient for all participants. The inclusion of design parameters was thus found to be effective in estimating the impression based on garment design parameters with LightGBM.