This paper proposes a method to perform 6-DOF SLAM by using scan matching based on the three-dimensional distance information acquired by omnidirectional stereo vision. Acquired scan is evaluated by the Iterative Closest Point (ICP) algorithm, then the result is integrated by the extended Kalman filter (EKF). Evaluation of the proposed method is done by the experiments on a simulation environment. The evaluation results show the effectiveness of the proposed method.