Proceedings of the Annual Conference of JSAI
Online ISSN : 2758-7347
34th (2020)
Session ID : 1E3-GS-9-03
Conference information

Automatic punctuation completion using LSTM
*Katsuhiko UTSUBO
Author information
Keywords: NLP, LSTM
CONFERENCE PROCEEDINGS FREE ACCESS

Details
Abstract

In Japanese sentences, the meaning of the context may differ depending on the insertion point of punctuation, so the position of the punctuation is very important. In this research, we create a general method that automatically complements punctuation from text information using deep learning. The proposed method is that the corpus is split using morphological analysis and replaced infrequent words with parts of speech and performs classification of exists of a period or comma using LSTM from word strings before and after the target position. The accuracy of classification has been improved by setting a threshold for the probability output by the model. Furthermore, by limiting the number of input words and replacing them with parts of speech, the calculation time can be reduced without reducing the calculation accuracy. Using this method, experiments using broadcast manuscripts as text corpora have confirmed the effectiveness of this method.

Content from these authors
© 2020 The Japanese Society for Artificial Intelligence
Previous article Next article
feedback
Top