GENERATING DAMAGE DESCRIPTIONS FROM BRIDGE IMAGES USING ATTENTION MECHANISM MODEL

Abstract

Various information such as types of damage and names of members are included in the image of the damage of the bridge. In this study, we developed a deep learning model that can automatically generate sentences describing the damage status based on images of various damages and members. In order to improve the accuracy, we incorporated an Attention mechanism into the Deep Learning model. In addition, when generating sentences, we generated a visualization image that shows which part of the input image the Deep Learning model was focused on. The model with the Attention mechanism was able to generate sentences with higher accuracy than the model without the Attention mechanism, and it output words by focusing on the location of the damage and each member when generating sentences. In addition, the model focused not only on the damaged member, but also on the surrounding members, suggesting that the model makes decisions based on a similar point of view as engineer.