Do LLMs Encode Frame Semantics? Evidence from Frame Identification
This addresses the problem of understanding semantic structures in natural language processing for researchers and practitioners, though it is incremental as it builds on existing FrameNet resources and model capabilities.
The study investigated whether large language models encode latent knowledge of frame semantics by evaluating their ability to perform frame identification without explicit supervision, finding they can do so effectively and improve with fine-tuning on FrameNet data.
We investigate whether large language models encode latent knowledge of frame semantics, focusing on frame identification, a core challenge in frame semantic parsing that involves selecting the appropriate semantic frame for a target word in context. Using the FrameNet lexical resource, we evaluate models under prompt-based inference and observe that they can perform frame identification effectively even without explicit supervision. To assess the impact of task-specific training, we fine-tune the model on FrameNet data, which substantially improves in-domain accuracy while generalizing well to out-of-domain benchmarks. Further analysis shows that the models can generate semantically coherent frame definitions, highlighting the model's internalized understanding of frame semantics.