Definition Generation for Semantic Frames with In-Context Learning

抄録

Semantic frame resources, such as FrameNet, rely on high-quality definition sentences to help humans intuitively understand the meaning of frames. Although recent research has focused on the automatic induction of semantic frames from large text corpora, the generation of frame definitions remains a task that requires high-level expertise and is costly. In this paper, we propose a novel approach to automatically generate frame definitions from clusters of frame-evoking words and their associated exemplars. Our approach leverages large language models (LLMs) to produce precise and coherent definitions, effectively reducing the manual effort required for this task. We found that traditional evaluation metrics often fall short in accurately assessing the quality of generated definitions. To overcome this challenge, we introduce an LLM-based evaluation framework that incorporates human annotations to identify the most effective evaluation prompts. Our experimental results show that in-context learning (ICL) can enhance the quality of the generated definitions. Moreover, incorporating contextually similar demonstrations within the ICL framework leads to further performance gains.