Person Search System using Clothing Features

Abstract

This paper proposes a person search system that uses clothing features. The system uses a camera to obtain the appearance of a person and a range sensor for person detection and direction estimation. The system detects a moving person based on background subtraction using distance information and histogram of oriented gradients (HOG) features. The clothing area is extracted according to the positions of the person's shoulders and waist. The clothes are characterized by four features that are calculated from the color histogram, color differences, frequency, and co-occurrence matrix. The system recognizes the same clothing based on the degree of similarity, which is obtained by comparing clothing features. However, two problems can be assumed: (1) the clothing appearance differs depending on the body direction and (2) the clothing color changes depending on the illumination. For the former problem, we prepared three databases (front/side/rear views) corresponding to the person direction. The system estimates the direction of the person by using information of the body arc based on ellipse fitting. The direction is determined as perpendicular to the long axis of the ellipse fitted to the cross-section of body. Concerning the latter problem, we calibrated the color information to correspond to the environment. The method recognizes clothes by selecting the database provided for the direction of the person; the results showed that it was robust at identifying specific persons.