An Analysis of Co-occurring Clusters Focusing on Topic Features of Documents using a Sparse Autoencoder

Abstract

With the advancement of large language models (LLMs), increasing attention has been paid to understanding their internal mechanisms. This study aims to clarify how LLMs internally represent topic information within documents. Based on the hypothesis that topics are represented as co-occurring features, we decomposed intermediate representations using a sparse autoencoder and clustered the resulting features according to their co-occurrence patterns. Analysis of the cluster characteristics revealed that they could be broadly categorized into those capturing grammatical features and those capturing conceptual features. Furthermore, by reclustering features with high topic-dependent activation frequencies, we observed a limited but consistent correspondence between clusters and document topics.