Portability of an eye tracking system encourages us to develop the technique for estimating 3D gaze position. Unlike the conventional methods, which estimate the position in the 2D image coordinates of the mounted camera, such a technique can represent richer gaze information of the human who moves in the larger area. In this paper, we propose a method for estimating 3D gaze position in the world coordinates by the head-mounted device. We employ visual SLAM technique to estimate head configuration and extract the environmental information. Even in the case where the head dynamically moves, the proposed method could obtain the point-of-regard.