Multimodal Prediction of Social Responsiveness Score with BERT-Based Text Features

Abstract

Social Skills Training (SST) has been used for years to improve individuals' social skills toward building a better daily life. In SST carried out by humans, the social skills level is usually evaluated through a verbal interview conducted by the trainer. Although this evaluation is based on psychiatric knowledge and professional experience, its quality depends on the trainer's capabilities. Therefore, to standardize such evaluations, quantifiable metrics are required. To meet this need, the second edition of the Social Responsiveness Scale (SRS-2) offers a viable solution because it has been extensively tested and standardized by empirical research works. This paper describes the development of an automated method to evaluate a person's social skills level based on SRS-2. We use multimodal features, including BERT-based features, and perform score estimation with a 0.76 Pearson correlation coefficient while using feature selection. In addition, we examine the linguistic aspects of BERT-based features through subjective evaluations. Consequently, the BERT-based features show a strong negative correlation with human subjective scores of fluency, appropriate word choice, and understandable speech structure.