Local Explanations With Consistency Constraints Between Instance and Feature Attributions for Set Functions

Abstract

In model-agnostic local explanations for set functions, two types of explanations can be considered: instance attributions (IAs), and feature attributions (FAs). It is natural to assume that IAs are consistent with the sum of FAs associated with that instance. Although such consistency is desirable from the viewpoint of explanation reliability and human interpretability, existing explanation methods are hard to achieve such consistency in practice. In this study, we propose a model-agnostic local explanation method to estimate IAs and FAs simultaneously under the consistency constraint of IAs and FAs. Experimental results show that the proposed method can achieve consistency between IAs and FAs, and produce high-quality explanations with a smaller number of queries to the set function.