Damage level estimation of inspection images in road infrastructures using in-context learning with data augmentation

Abstract

This paper proposes a damage level estimation model that incorporates in-context learning with data augmentation to achieve accurate classification of inspection images in road infrastructures, even with a limited amount of data. Conventional methods face challenges such as the ambiguity between damage levels in inspection images and the requirement for large-scale datasets to construct advanced models. In contrast, the proposed method utilizes a pre-trained Large Multimodal Model (LMM) and introduces incontext learning with inspection images using data augmentation (augmented images) to construct an LMM capable of damage level estimation. Furthermore, by integrating the estimation results of each augmented image, the method generates a final output and enables accurate damage level estimation even for challenging images. Finally, we validate the effectiveness of the proposed method using actual inspection images from road infrastructures by comparing its performance with other methods for damage level estimation.