Visualization of sounds from multiple musical instruments

Abstract

Human beings recognize external world through multiple sensory functions. In recent years, it is getting obvious that sensory functions are not independent but affect each other. Also, this phenomenon is called cross-modality. In this paper, I focused on cross-modality of auditory and visual sensation and investigated relationship between sound from a musical instrument and shape which is reminded from sound. Firstly, I prepared 10 sounds from 10 different instruments (Piano, Organ, Guitar, Violin, Cello, Harp, Trumpet, Clarinet, Flute, Saxophone) and investigated trends of the shape felt from sound by making subjects draw sketches. From this investigation I found some trends from sketches and made 25 planer shape samples. Secondary, I conducted a survey by using 25 samples which I described. As a result, I found similar trends from samples which are chosen for some instruments. Especially, Harp, Trumpet and Flute had obvious and really similar trends to previous trends from sketches. From these results, I made 40 three-dimensional shape samples and conducted same survey as planer shape samples. Finally, I found harp, trumpet and flute had obvious and similar trends through all the surveys.