BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
This work addresses the underexplored challenge of cross-lingual knowledge editing for multilingual AI systems, providing a comprehensive benchmark and insights into factors affecting generalization, though it is incremental as it builds on existing knowledge editing datasets.
The paper tackles the problem of cross-lingual knowledge editing, where edits in one language must generalize to others, by introducing BMIKE-53, a benchmark covering 53 languages. It finds that model scale and tailored demonstrations significantly improve performance, with larger models and aligned setups enhancing efficacy, while linguistic properties like script type cause performance variations, especially for non-Latin languages.
This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE) across 53 languages, unifying three knowledge editing (KE) datasets: zsRE, CounterFact, and WikiFactDiff. Cross-lingual KE, which requires knowledge edited in one language to generalize across others while preserving unrelated knowledge, remains underexplored. To address this gap, we systematically evaluate IKE under zero-shot, one-shot, and few-shot setups, incorporating tailored metric-specific demonstrations. Our findings reveal that model scale and demonstration alignment critically govern cross-lingual IKE efficacy, with larger models and tailored demonstrations significantly improving performance. Linguistic properties, particularly script type, strongly influence performance variation across languages, with non-Latin languages underperforming due to issues like language confusion. Code and data are publicly available at: https://github.com/ercong21/MultiKnow/.