Sound Target Tracking using an Omni-directional Camera and Two Microphones with a General State Space Model for Sensor Fusion using Particle Filters

Abstract

Sensor fusion aims at obtaining information, which cannot be obtained by single sensor, by combining signals from multiple sensors. The main problem of sensor fusion is huge computational cost increasing exponentially with the number of sensors since the combination number of associations between signals and sensors is large. A general method for this problem is proposed in a form of nonlinear state space model to deal with the unknown associations. The target states and the associations are simultaneously estimated through the state estimation. In the estimation of the associations, we apply particle filters with clever proposal. The associations are estimated in probabilistic way to avoid being trapped by a bad solution. We adapt the method to a specific situation of omni-directional camera and two microphones, to a sound target.