FPGA implementation of a neuro-inspired algorithm for spatio-temporal visual feature extraction

Abstract

Real-time extraction of visual features related to shape and motion is one of the most important functions for robotic visual systems because these features are required for various tasks such as object / action classification. In the present study, we developed a real-time image sensor system for spatiotemporal visual feature extraction based on a physiological model of the visual nervous system. The proposed system is composed of a CMOS image sensor, field-programmable gate arrays (FPGAs), and a personal computer. Digital circuits implemented in the FPGAs apply multiple spatio-temporal filters in parallel required for extracting visual features that are selective to direction of motion. We examined the system output by presenting a human waving his hands to the system, and the output exhibited good sensitivity to direction of motion.