Vision-based Vibration Pattern Recognition Based on Time-frequency Response Matching

Abstract

In this paper, we propose a novel method for dynamics-based recognition to inspect a specified vibrating pattern by computing time-frequency responses in brightness signals at all the pixels of a high-frame-rate video sequence. The time-frequency responses, which are the histograms of time-frequency components in brightness, can be calculated with short-term Fourier transform, and they are pixelwisely matched with that for a reference vibrating pattern to judge whether a specific vibrating pattern is observed or not. The effectiveness of our dynamics-based recognition method is demonstrated by showing several localization results for flying multicopters and insects, whose rotating or flapping frequencies were at hundreds of Hertz.