Multi-class Semantic Segmentation by Optimized Class Probability Matrix

Abstract

In semantic segmentation, where all pixels in an image are labeled, CNN is known to produce highly accurate results, and it has been applied to automated driving techniques. Cross Entropy Loss is often used for learning semantic segmentation, and Intersection over Union (IoU) is often used as its evaluation metric. In order to achieve more accurate prediction, loss functions that directly optimize the IoU have been studied in recent years. However, almost of the previous studies have shown the effectiveness only in the case of two-class segmentation, and few studies have confirmed the effectiveness in multi-class segmentation. In this study, we propose a new loss function that improves the accuracy of the IoU by optimizing the matrix composed of class probabilities. The effectiveness of the proposed method is confirmed by experiments on two-class and multi-class segmentation.