Construction of system 2 by human-in-the-loop based on question generation driven by System 1

Abstract

Many processes within machine learning models are a black box, and in most cases, their inference cannot be explained in a way humans can understand. This becomes a serious problem when implementing machine learning in domains requiring accountability.Wan et al. have proposed a method called NBDT, which generates a classification model with a tree structure of binary classifiers as each node from a deep learning model for multi-class classification, but it is unclear what kind of decision each node represents.In our work, we propose a method to reconstruct a machine learning model whose features used for judgment can be explained in natural language by incorporating a tree structure model constructed by NBDT into human-in-the-loop.The proposed method extracts only the nodes whose judgments are clear for humans and reconstructs a transparent machine learning model based on human annotation.Through crowdsourcing experiments, we show that it is possible to build machine learning models based on human interpretable judgments expressed by natural language.