Click-Through Rate Prediction with Confidence for News Headlines Using Natural Gradient Boosting

Abstract

News headlines in online news services play an important role in expressing the content of news articles and in stimulating users to click article pages. If their click-through rate (CTR) can be predicted, it will be useful as reference information for news editors to refine headlines. However, it is difficult to accurately predict the CTR because it is influenced by many external factors such as timing and popularity of news topics. In this study, we propose a method for predicting the CTR of news headlines with confidence using a regression model called natural gradient boosting (NGBoost) that predicts a distribution. In order to confirm the usefulness of our proposed method, we perform an evaluation experiment using the A/B test log of news headlines and discuss the relationship between confidence and prediction performance.