Automatic Literature Review Generation with Transformer Encoder-Decoder Model

Abstract

In this study, we aim to automatically generate survey papers by the Transformer encoder-decoder model to assist researchers in their surveys. As a preliminary effort, we attempted to generate chapters of survey papers by the existing Transformer encoder-decoder model, Longformer and Big Bird. As a result, we find some results where Big Bird correctly captures the topic to be explained and generate appropriate sentences. On the other hand, the model generates inappropriate sentences for the most part since it could not correctly determine the topic.