A Study on Improving Anomaly Detection Performance based on a Vision Language Model for Highway Facility Inspection Using Vehicle-mounted Camera Images

Abstract

AI-based inspection support using vehicle-mounted camera images is expected to contribute to highway maintenance. Recent studies have proposed anomaly detection methods for road facilities using a Vision Language Model (VLM), which jointly processes images and language. However, these methods face two major challenges: an increase in unnecessary anomaly candidates that need to be checked by inspectors, and missed anomaly candidates that should be detected. In this paper, we investigate methods to improve VLM-based anomaly detection for highway facility inspection using vehicle-mounted camera images. Specifically, we introduce three components: setting the minimum area threshold for crop image creation according to asset characteristics, applying square crop processing, and using in-context learning with an expanded set of abnormal examples. Experiments using vehicle-mounted camera images captured on actual highways showed that the proposed method contributed to reducing missed anomaly candidates compared with the conventional method. The results also demonstrate that the visual information required for detection varies depending on the anomaly type.