Robust Eye Contact Detection for Multi-Party Conversational Systems

Abstract

Eye contact detection method for multi-party conversational systems is proposed. Detecting eye contact is an important function for multi-party conversational systems because eye concatact is an essential cue for deciding whether system should respond to a user utterance or not. Accuracy of a conventional eye contact detector given a single frame decreases because of various noises (e.g. blink, facial expression change, etc.) occured in real-time environment. The eye contact detection method that utilizes multiple frames is proposed for solving this problem. The system extract the feature for detecting eye concact by CNN from the image contains both eyes of a target user. Then, the feature is given to LSTM with attention mechanism as an input. The result of the experiment conducted using the gaze data in multi-party conversation shows effectiveness of the proposed method.