Cross-Lingual Activation Steering for Multilingual Language Models
This addresses the problem of language imbalance in multilingual models for users of non-dominant languages, offering an incremental improvement through inference-time adjustments.
The paper tackled performance gaps between dominant and non-dominant languages in multilingual language models by proposing Cross-Lingual Activation Steering (CLAS), a training-free inference-time intervention that modulates neuron activations, resulting in average improvements of 2.3% in accuracy and 3.4% in F1 score on classification and generation benchmarks while preserving high-resource language performance.
Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant languages. Prior work attributes this gap to imbalances between shared and language-specific neurons in multilingual representations. We propose Cross-Lingual Activation Steering (CLAS), a training-free inference-time intervention that selectively modulates neuron activations. We evaluate CLAS on classification and generation benchmarks, achieving average improvements of 2.3% (Acc.) and 3.4% (F1) respectively, while maintaining high-resource language performance. We discover that effective transfer operates through functional divergence rather than strict alignment; performance gains correlate with increased language cluster separation. Our results demonstrate that targeted activation steering can unlock latent multilingual capacity in existing models without modification to model weights.