Abstract
BP (Back Propagation) learning algorithm based on Conjugate Gradient Method is an effective method for high-speed learning. This method has originally been introduced for the problem to minimize the quadratic function and is guaranteed to converge to it in the number of times equal to the search surface dimensions. It is also applied as it is to a generalized function like BP learning algorithm, because in this case too, it can approximate locally to a quadratic function. However, it fails to converge in the number of times equal to the search dimensions in this case, requiring the step to take the second approximation process after the search dimensions reached a certain number. This step is called "Restart". However, this method occasionally falls into local minimal depending on search surface, because it locally approximates a generalized function to a quadratic function. The cause is considered that it continues searching until its number of times reaches the conventionally determined number of times before the restart, even if the approximation to a quadratic function is low in precision ; therefore this restart method is considered to improve the search efficiency when the quadratic function approximation is low in precision. We therefore propose an efficient learning algorithm based on Conjugate Gradient Method. The method proposed here can decide whether the restart is needed according to new parameters which evaluate the quadratic approximation accuracy at every search point. We call the proposed method "An improved Conjugate Gradient Method".
