Quantitative analysis of singing expression and facial gestures using image recognition with artificial intelligence

Abstract

This study quantitatively analyzes facial gestures during singing using image recognition artificial intelligence to investigate their relationship to singing expression. In expressive singing, the mouth corners are consistently higher, the cheeks are lifted, and the lips are more open compared to nonexpressive singing. A temporal analysis of mouth corner height, aligned with the musical score, reveals that vowel articulation, especially the vowel /o/, affects mouth shape; however, expressive singing consistently maintains higher mouth corners. Moreover, head movements are more pronounced during expressive singing. These findings illustrate that singing expression affects facial and head movements, offering a quantitative framework for analyzing the richness of singing expression.