A Study on Developmental Artificial Neural Networks Integrating Multiple Functions with Variational Autoencoder

Abstract

In conventional deep learning, the network topology acquired by learning a specific task is transferred to a different task based on a single learned model that is expected to best fit the new task, as in fine-tuning or transfer learning. On the other hand, the organism is able to abstract multiple previously acquired experiences, retain them as knowledge, and combine them to adapt to a new task. To accomplish this, the network structure to be acquired requires an architecture that allows the fusion of network topologies with specific functions. Considering this, in this study, we aim to develop a Developmental Artificial Neural Networks (DANN) capable of simultaneously transferring multiple functions that generate a neural network with new functions that integrate multiple functions. we use the framework of Weight Agnostic Neural Networks (WANN) [1], in which the topology is obrtained evolutionarily rather than by the synaptic weight of the networks, and the acquired topology is transformed into a matrix by using an embedded representation of the generation rules, which is then used as input to the modified model of Grammar Variational Autoencoder (GVAE) [2]. Through the experimental results, multiple neural network models that solve individual tasks can be represented in the latent space, and correspondence between functions and latent representations is also observed.