日本語自然言語処理における事前学習モデルの公開

抄録

We have developed two types of pre-trained models, GPT-2 and RoBERTa, that are trained from a public corpus consisting of about 75-gigabyte texts. The models and its training code have been released under licenses that allow for commercial use. By fine-tuning the released models, users will be able to accomplish a variety of Japanese natural language processing tasks with high task accuracy.