SetKE: Knowledge Editing for Knowledge Elements Overlap
This addresses a specific bottleneck in knowledge editing for LLMs, making it incremental by focusing on overlapping elements to improve accuracy in updates.
The paper tackles the problem of knowledge editing in large language models when multiple knowledge triplets share overlapping elements, which causes conflicts and performance degradation in existing methods. They propose SetKE, a method that edits sets of triplets simultaneously, and show it outperforms current methods in such scenarios, while also introducing a new dataset, EditSet, for benchmarking.
Large Language Models (LLMs) excel in tasks such as retrieval and question answering but require updates to incorporate new knowledge and reduce inaccuracies and hallucinations. Traditional updating methods, like fine-tuning and incremental learning, face challenges such as overfitting and high computational costs. Knowledge Editing (KE) provides a promising alternative but often overlooks the Knowledge Element Overlap (KEO) phenomenon, where multiple triplets share common elements, leading to editing conflicts. We identify the prevalence of KEO in existing KE datasets and show its significant impact on current KE methods, causing performance degradation in handling such triplets. To address this, we propose a new formulation, Knowledge Set Editing (KSE), and introduce SetKE, a method that edits sets of triplets simultaneously. Experimental results demonstrate that SetKE outperforms existing methods in KEO scenarios on mainstream LLMs. Additionally, we introduce EditSet, a dataset containing KEO triplets, providing a comprehensive benchmark.