Correlation between topics in newspaper articles calculated by LDA and stock index

Abstract

We analyzed the correlation between topics in newspaper articles and stock indices to see if we could capture signs of economic changes from the text of the articles that report on various social events. First, we classified the words of 77,814 articles from a single Japanese newspaper over a 10-year period from 2003 to 2012 into 30 topics, and calculated the distribution of each topic on each day by Latent Dirichlet Allocation (LDA). Next, we calculated the rank correlation between the topics and the profitability and volatility of the 34 Tokyo Stock Exchange indices. As a result, we found that although there was no correlation between the topics and the profitability of stock indices, there was a significant time-dependent correlation between some topics and the volatility of the stock indices.