AI Quality Management Technologies in Operation: from Concept Drift Detection and Adaptation to Unsupervised Domain Adaptation

Abstract

In the operation of AI system using machine learning technologies, the quality and performance of the system may deteriorate due to changes of data in operation. In particular, the change in data distribution, called concept drift, is one of the main causes of performance degradation. In addition, in the operation of upcoming AI system, there would be the cases where the training data used before the operation cannot be reused because of their privacy and security concerns. In this paper, we arrange concept drift detection and adaptation technologies and unsupervised domain adaptation technologies that we have introduced in JSAI2020, JSAI2021, and JSAI2022 by showing their effectiveness to maintain quality and performance of the AI systems in operation. Furthermore, we also introduce the recent research on “Test-time adaptation techniques” which adapt machine learning models online without reusing training data, effectively addressing new problems that may arise during AI operation.