Tightening-state detection system for high strength bolts using mixed reality device -Application of YOLO and deep metric learning

Abstract

High strength bolts are used to combine the main parts of steel bridges. In the tightening management, inspectors visually check all the bolts after the final tightening step based on each bolt’s marking status. We have developed a system for automatically detecting the tightening states of high strength bolts in order to prevent inspection omissions, using images captured by a mixed reality device. In this system, images of high strength bolts are extracted from the captured image using YOLO, a deep learning object detection algorithm, and the distance from the normal state is detected using deep metric learning. We have applied a Laplacian filter to calculate the blurriness to exclude excessively blurred images, so that the detection will stay stable even during movement. In operational tests using real steel girders, we confirmed that the system accurately detected the tightening states and displayed the judgment results on the window of a mixed reality device in near real time even while moving.