Analysis of Communication Characteristics of Adults with ASD in Group Conversations

Abstract

Aiming at reducing communication difficulties between people with ASD and neuro-typical people, this study proposes deep neural network models that detect miscommunication in group discussions that people with ASD participate. First, we collected group conversation corpus among three people, including one diagnosed with ASD. Then, we defined "Miscommunication" by breaking down into two sub-categories: interaction issues (engagement, turn-taking, conversation context) and ASD specific issues. We annotated miscommunication based on these definitions, and used the annotation as ground truth in training machine learning models. We created multimodal transformer-based models using audio, facial, and language information, and found that the model performance of detecting miscommunication was 0.713 for F1-score and 0.652 for Accuracy. These results indicate the possibility of automatically detecting miscommunication in group discussions with people with ASD.