Automatic Consultation System by Latent Variable Hierarchical Recurrent Encoder-decoder using Transfer Learning

Abstract

In this paper, we propose an automatic consultation system by latent variable hierarchical recurrent encoder-decoder (VHRED) using transfer learning. VHRED has been developed to alleviate a large shortcoming of the sequence to sequence. They cannot consider the flow of dialog: the same output is produced for the same input. However, when we try to use VHRED, there is a small amount of Japanese corpus with long dialog-turns. The proposed system employs a method of transfer learning: the encoder layer and the decoder layer in VHRED are learned using a large corpus of dialog pairs obtained easily such as from Twitter and the other layers are learned by transfer learning using a small corpus with long dialog-turns. In the evaluation experiment, subjective evaluation experiments were carried out to compare with VHRED without transfer learning. As the results, it is shown that transfer learning of VHRED became possible by the proposed method.