Prototype of evaluation system for Chinese poetry generation

Abstract

　Tang poem is a well-established of Chinese culture and it is one of the indispensable resources in classical literature. Tang poem has unique stylistic and rhythmic rules. These rules make the reader immersive and feel more relevant to the author. When beginners learn a poem, it is too di!cult to observe the rules when they create poems, but also check whether the rules are right. This study aims to propose a poem generation and evaluation system. We used the existing pre-trained GPT-2 model and automatically generated Tang poems, then the generated poems can be judged by the evaluation system and got a ranked list which is conformed to the rules. We selected 8 random keywords from the Chinese poetry dictionary, and generated 100 samples of poems by each word, then evaluated which poems follow the rules. The results showed that our framework worked in generation and got good results from the evaluation system. As a result, of the 100 poems that were generated, 9.9% of generated poems got greater than or equal to 600 points, 16.8% of generated poems got greater than or equal to 500 points and less than 600 points, and 12.4% of generated poems got greater than or equal to 400 points and less than 500 points. On the other hand, 26.8% of generated poems did not match the writing styles. In the future, we plan to conduct evaluation experiments.