Context-Sensitive Generation of Narratives using Focus Vector

Abstract

In recent years, natural language processing technology utilizing large-scale language models has made remarkable progress, and the technology of sentence generation has also improved dramatically. Among these, BERT proposed by Google based on Transformer and GPT proposed by OpenAI are representative examples.However, even with these large-scale language models, it is said to be difficult to generate narrative sentences that have an attractive flow and an ending. Automatic generation of such narrative text requires a huge human cost, such as manually creating the plot and manually formatting the output.In this study, we propose a method to generate stories using a large language model GPT-2, with correction using focus vectors automatically obtained from existing children's stories, in order to generate stories with an ending without flow disruptions in an automated way.