Applying NLP for Automating Horizon Scanning

Abstract

We propose document retrieval and comment generation tasks for automating horizon scanning (Sardar 2010). The steps used are: 1) retrieving news articles that imply drastic changes of society, and 2) writing subjective comments on each article for others' ease of understanding. We manually collected articles to be retrieved and subjective comments. We compare several methods for two proposed settings. Our experiments show that 1) manually collected articles are different from general articles regarding the words used and semantic distances, 2) the contents in the comment can be classified into several categories. Our experiments showed that our BERT model achieves sufficiently high performance while the comment generation is challenging.