Application of Visual Servoing to Drawer Retrieval and Storage Operations

Abstract

The number of logistics warehouses is increasing due to the expansion of the e-commerce market. Among warehouse tasks, the retrieval, transportation, and storage operations of containers holding products on shelves are becoming increasingly automated because of labor shortages and the physical strain of workers. However, automated warehouse systems face challenges, such as the requirement for extensive space for implementation, high maintenance costs owing to rail-based positioning, and high pre-adjustment costs due to positioning with augmented reality markers or other signs attached to each container and shelf. To solve these problems, we aim to develop a system that enables the retrieval, transportation, and storage operations of containers without relying on positioning tools such as rails or signs. Specifically, we propose a system that uses a fork-inspired hand to handle a drawer equipped with fork pockets for retrieval and storage operations while considering the benefits of using drawers in terms of space efficiency and product management. For positioning, we use visual servoing with images obtained from a camera mounted at the end of the robot arm. In this study, we implement the following three methods in the proposed system and conduct comparative studies: image-based visual servoing (IBVS), which uses luminance values directly as features; active visual servoing (AVS), which projects patterned light onto an object for positioning; and convolutional neural networks-based visual servoing (CNN-VS), which infers the relative pose by using the difference between the feature maps of desired and current images. The experimental results show that CNN-VS outperforms the other methods in terms of the operation success rate.