Multi-Branch Neural Networks with Functional Localization by Branch Control

Abstract

Neural networks (NNs) can solve only a simple problem if the network size is too small. On the other hand, if the network size increases, it costs a lot in terms of memory space andcalculation time. Therefore, we have studied how to construct the network structure with high performances and low costs in space and time. A solution is a multi-branch structure. Conventional NNs use the single-branch for the connections, while the multi-branch structurehas multibranches between nodes. In this paper, a new method which enables the multi-branch NNs to have functional localization is proposed. Neural networks with Branch Control adjust signals propagating through branches between the intermediate layer and output layer depending on the inputs of the network. Therefore, a branch could be cut depending on input values. Simulation results of function approximations and a classification problem illustrated the effectiveness of multi-branch NNs with functional localization.