Intelligent Auditory Sensing using Nonlinear Loudness/Pitch/Timbre Decomposition Operators

Abstract

For extracting simple and informative measures for describing time-varying natures of musical sound and speech, we construct a wavelet energy distribution and orthogonally decompose instantaneous changing energy of it into three primary components: 1) loudness change, 2) pitch shift, and 3) timbre change according to the coherency in power magnification and frequency shift. The decomposition is performed in the time-frequency gradient space, and the power ratios of three components are transferred as signals to higher processing stages. Several experiments show that these signals have superior resolution and sensitivity for segmenting dynamical nature of speech, musical sounds, and so on.