Weighted Asymmetric Loss for Multi-Label Text Classification on Imbalanced Data

Abstract

Multi-label text classification, which assigns multiple labels to a single text, is a key task in natural language processing. In this task, a model is often trained on an imbalanced dataset whose label frequencies follow a long-tail distribution. Low-frequency labels that rarely appear in training data have an extremely small number of positive samples, so most of the input samples are negative. Therefore, the model learns low-frequency labels with the loss value dominated by the negative samples. In this research, we propose a method called weighted asymmetric loss that combines the appearance frequency weight of labels, the weight that suppresses the loss value derived from negative samples, and a label smoothing method in accordance with the co-occurrences of each label. Experimental results demonstrate that the proposed method improves the accuracy compared to existing methods, especially on imbalanced datasets.