Using Attention to Improve Explainability of Document Classification by BERT

Abstract

In recent years, many studies have been conducted on how to give explainability to deep learning models. BERT is a deep learning natural language processing model with excellent performance. Attention calculated internally by BERT can be used for explanation, but there is a problem that it is difficult for people to interpret the explanation. In this study, we examined the explainability of document classification in BERT, taking attention into account.