Journal of Information Processing
Online ISSN : 1882-6652
ISSN-L : 1882-6652
 
Automatic Stopword Generation Based on Attention for Document Classification Using Neural Networks
Yuki KuwabaraYu Suzuki
Author information
JOURNAL FREE ACCESS

2024 Volume 32 Pages 487-495

Details
Abstract

Stopwords are generally used to improve the accuracy of document classification and retrieval. We believe that setting appropriate stopwords improves classification accuracy. However, in our preliminary experiments, in document classification tasks using BERT, existing stopword lists are not effective for improving classification accuracy. To solve this problem, we construct a method for generating stopwords using the attention mechanism of the classifiers. In this method, words with high attention in misclassified input documents and low attention in correctly classified documents are treated as stopwords. The system probabilistically removes stopwords. The system automatically sets the probability of each word in input documents being a stopword when it builds the classification model. We conduct experiments to confirm effectiveness of our stopword generation method. Our experimental results show that there are cases using stopwords generated by our method that improve the classification accuracy. Three of the six classification tasks tested in this study show significant differences in accuracy improvement.

Content from these authors
© 2024 by the Information Processing Society of Japan
Previous article Next article
feedback
Top