Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey
For researchers working on multilingual knowledge editing, this work provides empirical guidance on the strengths and limitations of merging methods, though it is incremental in nature.
This paper investigates vector merging methods for multilingual knowledge editing (MKE) in large language models, finding that vector summation with shared covariance is the most reliable strategy, while TSVM offers limited interference reduction. Performance is sensitive to weight scaling and rank compression, with larger scaling and lower rank often improving results.
Multilingual knowledge editing (MKE) remains challenging because language-specific edits interfere with one another, even when locate-then-edit methods work well in monolingual settings. This paper focuses on three issues: the effectiveness of vector merging methods for MKE, the extent to which Task Singular Vectors for Merging (TSVM) can reduce multilingual interference, and the influence of the weight scaling factor and rank compression ratio on performance. We evaluate six merging variants with two popular backbone large language models, two base knowledge editing methods, and 12 languages on the MzsRE benchmark under a large-scale batch-editing setting. Our results show that vector summation with shared covariance is the most reliable overall strategy, whereas simple summation without shared covariance performs poorly. TSVM improves performance in some settings, but its ability to mitigate multilingual interference is limited. We also find that performance is sensitive to both weight scale and rank ratio, with larger-than-default scaling and relatively low rank often yielding better results. These findings clarify the practical strengths and limits of current vector merging methods for MKE and provide guidance for future multilingual knowledge editing research.