Validity Judgment of Phrase Connectivity by Self-Supervised Learning for Change Point Detection

Abstract

The recent rapid development of Deep Neural Networks (DNNs) has led to various technological innovations in Natural Language Processing (NLP). However, DNNs require a large amount of training data, and labeling supervised signals is the bottleneck in training data generation. For this reason, self-supervised learning (SSL), which generates supervised training data from unsupervised training data, has been attracting attention. On the other hand, there has been extensive research on proofreading support for Japanese texts, enabling the detection of superficial errors such as spelling and homonym errors. This study proposes an SSL-based method for validity judgment of phrase connectivity based on grammatical or semantic integrity. The proposed method synthesizes supervised training data by cutting and connecting two randomly selected phrases and assigns ground truth labels. Experimental results demonstrated the effectiveness of the proposed method in the NLP task.