Selective Learning Algorithms with Forgetting Taking Account of the Balance between Learning and Forgetting

Abstract

In this paper, the relation between the correction and the forgetting of weights in the learning algorithm is examined and selective learning algorithms with forgetting are newly proposed. First, the relation between the direction of the correction and the direction where the weight is forgotten is examined and a learning algorithm with forgetting based on the learning error is proposed. Concretely, the weight is selectively forgotten by introducing the index to evaluate the learning ability. Next, a learning algorithm with forgetting based on the activity of the hidden unit is proposed in order to improve the structuring ability. Finally, the comparison examination of such learning algorithms with forgetting is done from the viewpoint of the learning ability, the structuring ability, and generalization ability by using the logical function learning problem, the classification problem of irises, and the time series prediction problem and the effectiveness of the proposed learning algorithm with forgetting is shown.