News Headline Generation Based on Reinforcement Learning Considering Human Evaluation

Abstract

In online news services, users look at a headline, and determine whether to read its article. Therefore, presenting attractive headlines is important to increase the number of article views. An encoder-decoder model makes it possible to generate natural headlines. However, a widely-used objective function is a token level, and thus the goodness of an entire heading is not considered. This paper proposes a method for generating headlines based on reinforcement learning, considering the human evaluation of an entire headline. First, we evaluate which headline is better by crowdsourcing, headlines by editors or headlines by a baseline system, and we train a headline-pair evaluator using these data. Then, a headline generation model is trained based on reinforcement learning where the prediction of the evaluator is used as a reward. Experimental results demonstrated that the proposed method could generate better headlines in comparison with several baseline systems.