This paper presents a multi-agent based method to extract the same objects in image sequences. In the proposed method, the objects are divided into line segments which are considered to be agents. Further, the method is based on a multi-agent algorithm of which the utility function is defied with a motion continuity and a shape classification of line segments. By optimizing the utility, the proposed method searches for a suitable line segment which corresponds to the target segment in a different frame. According to the group rationality, the proposed method can correctly provide correspondence among line segments in different frames. Further, the computational cost can be reduced by utilizing the group rationality. The proposed method can also extract occluded objects by tracking utilities in multi frames.