Utterance Classification in Motivational Interviewing using Verbal, Facial, and Speech information

Abstract

Motivational interviewing (MI) is a counseling technique that aims to elicit clients' reasons for behavior change. In MI, a coding scheme called Motivational Interviewing Skill Code (MISC) has been established. In this study, we first annotated counselor utterances in a Japanese MI corpus using MISC coding scheme, and merged the labels into 13 categories. Then, we created 13-class classification models using two approaches. The first approach is to create classification models by fine-tuning a large-scale language model (LLM). The second approach is to create cross-modal transformer models based on BERT. Experimental results showed that the best F1-score was 0.83 for complex reflection category, which includes summaries and metaphors. We also discussed the impact of unbalanced data.