Proposal for the Application of an Embedding Representation Model Specialized in the Field of Civil Engineering

Abstract

In recent years, the technology of large language models (LLMs) has been advancing, but there are chal lenges such as the lack of knowledge in specific fields. To address this issue, retrieval-augmented generati on (RAG) has been gaining attention. Many RAG technologies are realized through embedding representa tion models, but there are challenges in customization and cost-effectiveness for specific fields such as civ il engineering. In this study, we constructed customizable models and small-scale models for adaptation to the civil engineering field through contrastive learning on OpenAI’s text-embedding-3-large. The accurac y verification results confirmed the effectiveness of the models in adapting to the civil engineering field a nd enabling small-scale, fast computation.