Associative Memory with Pattern Analysis and Synthesis by a Bottleneck Neural Network(<Special Issue>Contribution to 21 Century Intelligent Technologies and Bioinformatics)

Abstract

We propose a new associative memory to improve its noise tolerance and storage capacity. Our underlying model is an improved multidirectional associative memory (IMAM), which uses autoassociative bottleneck neural networks to remove noise in its input, i.e., analyze patterns. IMAM has inefficient storage capacity and low noise tolerance due to a correlation matrix representing association. One of our basic ideas is to replace a correlation matrix with a multilayer perceptron (MLP), which has better learning and generalization capability. Moreover, we introduce two improvements. One is to add intermediate elements into MLP to improve its performance. The other is to use outputs of hidden layers in a five-layer bottleneck neural network. These outputs include information on synthesis of a key pattern from compressed information in the middle layer. To evaluate the proposed approaches, we compared three types of associative memory: associative memory with a bottleneck neural network and MLP (AM/B-M), AM/B-M with intermediate elements (AM/B-I), and AM/B-I with synthetic outputs (AM/B-IS). 10-by-10 images of Latin alphabet are used as patterns for association. In a case of association between 78 non-injective pattern pairs with 10% noise, our proposed AM/B-IS is better than AM/B-M by more than 40% in pattern recalling ratio.