Improve Discontinuous Output Change in SpikeProp

Abstract

We try to improve discontinuous output change in SpikeProp. The problem is that small variants in input cause significant change in output. We first show that peaks in activity cause the problem. We then propose reducing the height of peaks by weight decay. Through experiments, we conclude that the square type of weight decay is suitable for solving the problem, because it reduces to more than half the patterns that cause discontinuous output change around them.