Evaluation of Knowledge Graph Construction Methods from Academic Abstracts Based on Large Language Models

Abstract

Traditional Knowledge Graph (KG) construction from complex academic texts relies heavily on human effort. This study investigates the potential of Large Language Models (LLMs)to automate this process.We comparatively evaluated three LLM-based KG methods (LLMGraphTransformer, KGGen, rahulnyk KG) using eight CiNii abstracts. Generated triples were assessed by experts for syntactic and semantic correctness.The results show LLM-based construction is promising but requires enhanced accuracy and robustness. We conclude that the choice of methodology must be strictly aligned with application requirements.