Visualization for Interpreting Classification Models by Comparing Multiple Local Explanations

Abstract

The reliability of machine learning models are essential when using models to make decisions in the real world. Various approaches have been proposed to make the models more interpretable and the prediction results more trustworthy. Among them, methods that regard the models as a black box and provide reasons for predictions are effective especially for users who have little knowledge of machine learning, such as domain experts and end users. In this paper, we propose a visual analysis method to support users in evaluating and improving the reliability of a model by utilizing model-agnostic explanation methods. Specifically, we adopt multiple local explanation techniques which explain individual input data and generate local explanations for a large set of data points and visualize them in a heat map. It reveals features that affect a wide range of the model. We applied our visualization to a diabetes classification model and verified its effectiveness in evaluating the trustworthiness of the model.