Estimating heterogeneous treatment effects of content recommendation using machine learning in ABEMA

Abstract

The video streaming service ABEMA conducts daily verification through A/B testing for the purpose of improving its services. In this paper, using the data from the A/B tests, not only the average treatment effect commonly dealt with in traditional effect verification was estimated, but also the heterogeneous treatment effect (HTE) was estimated using machine learning. As a result, it was confirmed that heterogeneity in treatment effects occurred depending on the trend of the content viewed before the experiment, and implications regarding user behavior were obtained.