Modelling Synthetic Non-Linguistic Utterances for Communication in Dialogue

Abstract

Non-Linguistic Utterances (NLUs) present a potentially useful additional communication channel between humans and machines. NLUs may be appropriate in situations such as assisting tourists with various language backgrounds and needs. An experiment was done to model NLU interpretation as dialogue parts. 1000 sounds were generated and evaluated for dialogue relevance to reduce the number to 53, used in the NLU evaluation experiment. 31 Subjects listened to the sounds and rated the following communicative acts for each on a 5-point scale: Greeting, Reject, Question, Thanking, Accept, Apology, Non-Understanding, and Exclamation. Factor analysis yielded three factors: 1) Interrogative vs Apologetic, 2) Appreciative vs Negative, and 3) Positive vs Non-Understanding. Prosodic features such as average pitch, pitch contour, timbre, and duration influenced the interpretation of the sounds. Future work will apply spectral-based speech and emotion recognition techniques along with prosodic features to allow machines to augment dialogue with effective NLUs.