Semantic grasping of everyday objects using a knowledge model

Abstract

This paper presents semantic grasping of everyday objects using a knowledge model. The knowledge model of an everyday object describes the structure of the object and information on how to manipulate it. Information on the structure of an object includes component parts of the object in terms of simple shape primitives to provide geometrical information and the connections between parts. Some everyday objects have movable connections between parts, so their kinematic attributes are described for those connections. The manipulation knowledge of the object includes the grasp areas and approach directions for the intended tasks. The grasp areas and approach directions are mapped on to the 3D point cloud data of the target using the geometrical information which is described in the knowledge model when the 3D point cloud data of the target and the intended task are given. Fine grasps suitable for the intended task are generated by performing a grasp planning. In the grasp planning, the system selects the appropriate operation according to the kinematic attributes of the objects and evaluates the efficiency for the task. We conduct experiments on semantic grasping for picking task and opening the lid of two types of container using the knowledge model and show the feasibility and the efficiency of our approach.