Bayesian Generalization Error in Three-layered Linear Neural Network with Concept Bottleneck Structure and Comparison with Multi-task Formulation

Abstract

Since responsibility of information systems using artificial intelligence has been needed,explanation methods have been proposed for machine learning models.Concept Bottleneck Model (CBM) is one of these methods for neural networks.In CBM, concepts which correspond to reasons of outputs are inserted in the last intermediate layer as observed values.It is expected that we can interpret the relation between the output and the concept like linear regression.However, it has not been clarified how the generalization performance of CBM changes from a neural network that concepts are not inserted.In this paper, we reveal the Bayesian generalization error in three-layered neural network with concept bottleneck structure.Besides, we compare it with that of multitask formulation which makes concepts co-occur with outputs.