Investigation of Embedded Text Communication with Onomatopoeia of User's Bodily Motion and Environmental Sounds

Abstract

In this paper, we propose an arranged text communication with the user's text messages and onomatopoeic texts of both the user's bodily motion and ambient sounds. The bodily motion is the user-subjective information, and the ambient sound is one of environmental information. The text communications have limitations on their expressions compared to other media such as sound, images or video. Instead of other modalities, the proposed method adopts onomatopoeic texts converted from the user's bodily motion and the environmental situation so that nonverbal expressions including the user's underconscious or uncontrolled information but without privacy information can be expressed in the text communication. We evaluated the different types of the expressive modality for pala-linguistic information in parallel with text communications. From the results, it is confirmed that onomatopoeic texts attached to the text communication can enrich communication information.