LANGSAE EDITING: Improving Multilingual Information Retrieval via Post-hoc Language Identity Removal
This addresses the issue of language bias in multilingual information retrieval for users needing cross-language search, though it is incremental as it builds on existing embedding methods.
The paper tackled the problem of multilingual dense retrieval where language identity in embeddings biases similarity towards same-language pairs, by proposing LANGSAE EDITING to remove language-identity signals post-hoc, resulting in consistent improvements in ranking quality and cross-language coverage, with strong gains for script-distinct languages.
Dense retrieval in multilingual settings often searches over mixed-language collections, yet multilingual embeddings encode language identity alongside semantics. This language signal can inflate similarity for same-language pairs and crowd out relevant evidence written in other languages. We propose LANGSAE EDITING, a post-hoc sparse autoencoder trained on pooled embeddings that enables controllable removal of language-identity signal directly in vector space. The method identifies language-associated latent units using cross-language activation statistics, suppresses these units at inference time, and reconstructs embeddings in the original dimensionality, making it compatible with existing vector databases without retraining the base encoder or re-encoding raw text. Experiments across multiple languages show consistent improvements in ranking quality and cross-language coverage, with especially strong gains for script-distinct languages.