Studies on association characteristics between vowels and visual colors using multiple speakers' speech

Abstract

Aiming at scientific understanding of color association after listening to vowel sounds, cross-modal association characteristics was investigated using parameters directly extracted from speech and color. A perceptual experiment was conducted by employing five Japanese vowel sounds produced by multiple male and female native speakers differing in spectral characteristics. For color description, three attributes (hue, saturation and value) of 153 color tips of Practical Color Coordinate System (PCCS) selected after listening to vowel speech were employed. To enable scientific interpretation, acoustic features of vowel sounds (F0, pitch range, intensity, F1, F2, F3, F4) were used. Regression analyses were carried out between these speech and color parameters. From the results using multiple linear regression and neural networks, significant correlations could have observed between F1, pitch range & F2 and hue, F0 & F2 and value of color. These experimental findings can exactly and clearly show vowel-color association characteristics which have been partially studied in phonetics and cognitive science fields mainly using speech and color categories. These precise parametric correlations between color and speech can provide scientific knowledge for further investigations of cross-modal correlations in multiple research fields which will give the new possibilities for multi-modal information expression.