Evaluation and Improvement of Domain Generalization Methods for Open-Set Recognition in Domain Shift

Abstract

In real-world applications, a machine learning model is required to handle an open-set recognition, where unknown classes appear during the inference, in addition to a domain shift, where the distribution of data differs between the training and inference stages. Domain-Augmented Meta-Learning (DAML) is a method to consider this situation, where both domain shift and open set recognition occur, but it has a complex learning process. On the other hand, although various domain generalization methods have been proposed to deal with domain shift, they have not been evaluated on open-set recognition in domain shift. This work comprehensively evaluates domain generalization methods for open-set recognition in domain shift and shows that two simple and computationally inexpensive domain generalization methods, CORrelation ALignment (CORAL) and Maximum Mean Discrepancy (MMD), exhibit comparable performance with DAML. In addition, we attempt to improve CORAL and MMD by introducing the techniques used in DAML, such as ensemble learning and data augmentation, and report their performance.