We present an efficient method to fixate a binocular gaze point on an object moving around in a complicated environment. Assuming the object is in the vicinity of the gaze point, the image features of the object can be isolated from others by using zero disparity filtering (ZDF). The object position in the depth direction, however, can not be estimated from such isolated features. In order to get the depth information simultaneously with the isolation by ZDF processing, we introduce the novel localization technique based on the idea of “virtual horopter”. The proposed method is implemented on our active vision system with much reduced programming efforts and resources in comparing with conventional methods. Several experiments demonstrate that the binocular tracking based on the proposed method works for various shape objects and in complicated environments.