Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models
This addresses safety concerns for users of multilingual AI systems by enabling selective knowledge removal across languages, though it is incremental as it extends unlearning techniques from monolingual to multilingual contexts.
The paper tackles the problem of machine unlearning in multilingual language models, where sensitive information can persist in less dominant languages, and presents an adaptive unlearning scheme that effectively erases information across languages while maintaining performance, achieving new standards in security and adaptability.
Pretrained language models memorize vast amounts of information, including private and copyrighted data, raising significant safety concerns. Retraining these models after excluding sensitive data is prohibitively expensive, making machine unlearning a viable, cost-effective alternative. Previous research has focused on machine unlearning for monolingual models, but we find that unlearning in one language does not necessarily transfer to others. This vulnerability makes models susceptible to low-resource language attacks, where sensitive information remains accessible in less dominant languages. This paper presents a pioneering approach to machine unlearning for multilingual language models, selectively erasing information across different languages while maintaining overall performance. Specifically, our method employs an adaptive unlearning scheme that assigns language-dependent weights to address different language performances of multilingual language models. Empirical results demonstrate the effectiveness of our framework compared to existing unlearning baselines, setting a new standard for secure and adaptable multilingual language models.