Study of Music Visualization Design using Musical Information

Abstract

As part of music visualization research that we are working on, we have been studying designs to visually feel the music structure of a homophonic song consisting of a main melody and accompaniment. In this paper, we propose a method for generating the images that describe the main melody and accompaniment individually. In order to express the information contained in music visually and naturally, we used knowledge of cross-modal correspondence in the design of the visualization and produced visualization content by synthesizing it with a video of a pianist's performance.