AdaSniper: An Adaptive Automated Approach for Systematic Error Detection in Image Recognition Models

Abstract

Image recognition models based on machine learning can exhibit systematic error tendencies under specific conditions. For instance, if a model consistently fails to correctly recognize a truck in images under the condition of a ``white truck in broad daylight," this can be considered a systematic error. Such errors can lead to unexpected behavior for users, making it crucial to detect systematic errors in advance through testing. Existing research has proposed methods that efficiently detect systematic errors by assisting human adaptive exploration. However, for machine learning models that require continuous and extensive testing, these methods pose significant human cost issues. In this study, we propose a method called AdaSniper, which utilizes large language models (LLMs) to automatically perform adaptive exploration and efficiently detect systematic errors. The proposed method enables adaptive exploration by providing the LLM with information on misclassified target classes, indicating which class the model incorrectly recognized. Evaluation experiments demonstrated that AdaSniper could perform adaptive exploration and efficiently detect systematic errors compared to baseline methods.