Exploring knowledge transfer graphs by introducing design space refinement method with human-in-the-loop

Abstract

Deep collaborative learning is a method of transferring knowledge between multiple networks. Knowledge transfer graph has been proposed as deep collaborative learning that makes a rich in diversity of knowledge transfer. However, designing a knowledge transfer graph is difficult due to many combinations, so it is not clear the trend for highly accurate knowledge transfer graphs. To address this problem, we propose a method for designing search space with human-in-the-loop for knowledge transfer graph. We analyze the trend of graphs and designing graphs with high accuracy based on the acquired results. The experimental results with CIFAR-100 show that the search space explored by the proposed method is better than that of deep mutual learning. We confirmed that the accuracy of the best knowledge transfer graph in the search space is better than that of using the asynchronous successive halving algorithm.