Knowledge Graph Enhanced Large Language Model Editing
This addresses the issue of limited generalization in LLM editing for NLP applications, though it appears incremental as it builds on existing model editing methods.
The paper tackles the problem of inaccuracies and outdated knowledge in large language models (LLMs) by proposing GLAME, a model editing method that uses knowledge graphs to track and incorporate associated knowledge changes, resulting in significantly improved generalization capabilities for post-edit LLMs as demonstrated in experiments on GPT-J and GPT-2 XL.
Large language models (LLMs) are pivotal in advancing natural language processing (NLP) tasks, yet their efficacy is hampered by inaccuracies and outdated knowledge. Model editing emerges as a promising solution to address these challenges. However, existing editing methods struggle to track and incorporate changes in knowledge associated with edits, which limits the generalization ability of postedit LLMs in processing edited knowledge. To tackle these problems, we propose a novel model editing method that leverages knowledge graphs for enhancing LLM editing, namely GLAME. Specifically, we first utilize a knowledge graph augmentation module to uncover associated knowledge that has changed due to editing, obtaining its internal representations within LLMs. This approach allows knowledge alterations within LLMs to be reflected through an external graph structure. Subsequently, we design a graph-based knowledge edit module to integrate structured knowledge into the model editing. This ensures that the updated parameters reflect not only the modifications of the edited knowledge but also the changes in other associated knowledge resulting from the editing process. Comprehensive experiments conducted on GPT-J and GPT-2 XL demonstrate that GLAME significantly improves the generalization capabilities of post-edit LLMs in employing edited knowledge.