Instance Segmentation Model Retrieval using Performance Prediction and Learning to Rank

Abstract

In this research, we propose a method to efficiently retrieve instance segmentation models that perform well on a specific image. With the advancement of machine learning and computer vision technologies, image instance segmentation has found widespread applications in industrial domains such as quality inspection, automated manufacturing, medical diagnosis, and autonomous systems. However, obtaining optimal results requires selecting an appropriate model from numerous candidates, which demands substantial time and computational resources as each model must be individually tested on target images. Our approach addresses this challenge by embedding candidate models and target images into high dimensional vector spaces and leveraging these representations to predict performances, without actual inference. We combine performance prediction with the Learning to Rank technique to accurately model the relative performance relationships among candidate models. The proposed method effectively identifies appropriate models for a specific image while eliminating the computational burden, offering a practical solution for real-world applications where computational efficiency is critical. To evaluate our method, we constructed an instance segmentation dataset containing 20 diverse categories, each with 20 images featuring varying object shapes, colors, and sizes. Experimental results using cross-validation demonstrate that our approach significantly outperforms baseline methods.