Evaluation of Knowledge Graph Transformation and Input Methods for LLMs ~ An Investigation through QA Tasks

Abstract

In this study, we investigated how transforming a knowledge graph and varying input formats affect QA tasks as a way to effectively integrate external knowledge into LLMs (large-scale language models). Specifically, using a QA dataset generated from the DBpedia infobox, we compared five different input methods, including natural language and JSON formats. The results revealed that a clear key-value structure facilitates the accurate extraction of required information by the LLM. On the other hand, approaches employing natural language formats or containing excessive information tended to show reduced accuracy due to redundancy and missing information.