Optimization of Integral Path in Explainable AI with Model Uncertainty for Suppressing Fluctuations in Explanatory Contents

Abstract

Uncertainty of XAI output is a serious problem for AI applications. In IG (Integrated Gradients), a typical XAI method, such high uncertainty occurs when the integration path used in IG passes through region with high uncertainty of AI in the feature space. The technical challenge is that we should change the path to avoid such high-uncertainty region while the path is uniquely determined as a straight line to satisfy the axioms of explanation. To address this challenge, we propose an extended IG framework where the integral path is composed of multiple paths. To evaluate the proposed method, we consider the artificial data setup where conventional IG has high uncertainty due to the nonlinear data distribution. As the result, we confirm that the proposed method can suppress the uncertainty of the output by 14% compared with the conventional one.