Mutual Gaze Detection for Human Interaction Analysis

Abstract

Human relationship recognition is an important research branch of human behavior analysis, which focuses on interaction patterns and the social relations among two or more persons. We hypothesize that human interactions, the frequency and length of mutual gazes can be used to infer human relationships through video analysis, particularly videos taken by security cameras. With the advancement of deep learning, there has been significant research progress in human interaction recognition for video surveillance systems. However, mutual gaze detection in surveillance camera’s views, in which faces cannot always be seen, still remains a challenge. To verify a part of our hypothesis, we conduct experiments to detect human mutual gazes in videos taken by security cameras and propose a system for mutual gaze detection built on top of a gaze estimator. Two deep learning-based approaches and one angle-based method are evaluated on the collected dataset to compare the effectiveness of the methods.