Development of Damage Diagnosis VQA Using Images and Comments in Bridge Inspection Reports

Abstract

In the damage diagnosis of bridge inspections, a comprehensive judgment is made from the inspector's professional viewpoint as well as objective facts such as damage conditions. In this study, we extracted information from inspectors' findings, which are considered to describe the basis for such judgments in damage diagnosis, and used a dataset created based on the extracted information to construct a deep learning model that can answer questions related to the diagnosis of various components and damage in images. The model was able to not only identify the name and type of damage, but also predict the state, cause, and inpact of the damage. Furthermore, the analysis of the model's evaluation and the basis for judgment suggested that the model made judgments based on criteria like those used by humans, such as paying attention not only to the member where damage occurred but also to the surrounding members.