ACCURACY VALIDATION OF PRE-TRAINED BERT IN THE CIVIL ENGINEERING DOMAIN

Abstract

 In recent years, natural language processing technology has made great strides with the advent of BERT, but it has been pointed out that there are issues in applying BERT to specialized fields where the amount of data is small.The Ministry of Land, Infrastructure, Transport and Tourism Data Platform has developped data aggregation platform that enables various types of data to be collected and utilized to promote various types of innovation and improve efficiency.However, the accuracy of existing natural language processing technology may not be sufficient for the specialized texts that we use in our daily lives, such as construction information, patrol inspections, and technical information in the civil engineering domain.We constructed “civil engineering BERT” by training BERT on sentences related to civil engineering.We verified the accuracy of the constructed “civil engineering BERT” and the existing BERT, and showed the superiority of "civil engineering BERT", and confirmed that training civil engineering sentences is effective.