Neocognitron: Deep convolutional neural network

Abstract

Deep convolutional neural networks (deep CNN) show a large power for robust recognition of visual patterns. The neocognitron, which was first proposed by Fukushima (1979), is recognized as the origin of deep CNNs. Its architecture was suggested by neurophysiological findings on the visual systems of mammals. It acquires the ability to recognize visual patterns robustly through learning. Although the neocognitron has a long history, improvements of the network are still continuing. For example, learning rule AiS (add-if-silent) for intermediate layers, learning rule mWTA (margined WTA) for the deepest layer, pattern classification by IntVec (interpolating-vector), a method for reducing the computational cost of IntVec without sacrificing the recognition rate, and so on. This paper discusses the recent neocognitron, focusing on differences from the conventional deep CNN. Some other functions of the visual system can also be realized by networks extended from the neocognitron, for example, recognition of partly occluded patterns.