Learning Rules of Neural Networks Using Time Difference Simultaneous Perturbation

Learning by only one value of an error function

Abstract

In this paper, we describe learning rules by means of the time difference simultaneous perturbation method. This recursive method was proposed by the first author to find a maximum or minimum of unknown functions by using only one value of the objective function at each iteration. We applied this to learning of neural networks. This approach is simpler than the well-known backpropagation method in the point that our rule needs only one value of the error function instead of the complicated derivation of derivatives in the backpropagation method. Some simple numerical examples are shown.