On Composite Neural Networks

Abstract

In this paper, we consider neural networks with three layers, where each unit itself is allowed to be a neural network with learning completed. (In the follwing, such neural networks are referred to as composite neural networks.) Existing neural networks are constructed for a single purpose, so that they cannot be used for more complicated information processing without teaching them from the beginning, while composite neural networks reuse these neural network resources. In composite neural networks, network units are connected by new synapses and only these weights are subject to updating through learning. This paper presents learning algorithms for composite neural networks with and without feedback in the middle layer. Two numerical examples are shown in the fields of logical circuit and alphabet recognition. Methods of interpolation learning, which is important for composite neural networks, are also given for each example.