Abstract
Speed-up techniques for backpropagation (BP) by removing singular points are simple and powerful, and therefore they are practical. We studied the techniques to secure generalization ability by computational experiments. We introduced “useful region” for gains of the speed-up techniques, where generalization ability is equivalent to that of the standard BP. Two kinds of experiments were conducted to test the speed-up techniques. One experiment applied the speed-up techniques to modify weights between hidden and output nodes (Case 1). The other applied the techniques to all weights of neural networks (Case 2). The results of the experiments found no useful region in Case 2. In Case 1, however, experiments found useful region when gains of the speed-up techniques were small.
