Distress estimation of road attachments based on attention-based multiple instance learning considering the diversity of background of images

Abstract

In this paper, we propose a distress estimation method for road attachments. Road attachments, such as signs and lighting, are equipped over a huge number and wide area, and therefore it is desired to achieve automatic inspection by using drones to reduce the burden on the inspectors. While captured images by drones include the diversity of the background including ground, sky, and road surfaces, the previous methods did not consider the diversity of the background of captured images of road attachments. This study proposes the distress estimation method via attention-based multiple instance learning to address this issue. We input patches of the images into the estimation model to distinguish between the background area and road attachment area and assign importance weight, or attention, to each patch. By performing this strategy, we realize the distress estimation method considering the diversity of the background area of images. In the experiment, we achieve a classification accuracy of about 70 % using images of actual road attachments confirming the effectiveness of this research approach.