Dynamic Structured Neural Topic Model with Self-attention Mechanism

Abstract

In this study, we aim to create a topic model that takes into account time-series transitions of topics and multi-year dependencies between topics. As a part of this effort, we show the limitations of the existing model, Dynamic Embedded Topic Model (D-ETM) and propose Dynamic Structured Neural Topic Model (DSNTM). DSNTM is based on D-ETM, while introducing a self-attention mechanism to represent the relationship between topics. After explaining the specific architecture of DSNTM, we discuss the current challenges and future prospects of our study.