Automatic generation of findings using generative AI to support for inspection report creation －Introduction of in-context learning based on similar image retrieval through cluster analysis－

Abstract

This paper proposes an automatic findings generation technique based on a Multimodal Large Language Model to enhance the efficiency of bridge maintenance management. The generative AI learns the relationship between distress images and their findings from a few examples and generates findings for desired distress images. To effectively leverage this capability, the proposed method employs clustering to identify the most informative inspection information for each findings category in learning task, and selects past inspection records based on this information. This approach not only improves the accuracy of the generated findings but also reveals the connection between the inspection information identified through clustering and the implicit knowledge in engineers’ findings creation Experiments verify that the proposed method successfully reflects the insights of engineers and achieves high-precision findings generation.