A study on application of large-scale language models to maintenance and management for infrastructure

Abstract

In recent years, large-scale language models (LLM) such as OpenAI's ChatGPT have been expected to be used. By using the trained model of LLM, various tasks such as question answering, text generation, translation, summarization, text classification and evaluation can be applied. Although LLM has learned a lot of general texts in advance, As a method of using specialized knowledge, there is a method of extracting features of the target text with LLM and having LLM perform the task again using the features, whereas there is a method of relearning the LLM model itself using original data (fine tuning). In this paper, we will briefly organize the LLM model and organize how to fine-tune the text in the civil engineering field. Using open data such as inspection procedures, we actually performed fine tuning, and from the results, we organized the issues from the viewpoint of training data, utilization of existing models, and evaluation method of generation results.