Learn and Unlearn: Addressing Misinformation in Multilingual LLMs
This addresses safety and reliability issues in multilingual LLMs for users across diverse linguistic contexts, representing an incremental improvement by highlighting a specific limitation in existing unlearning techniques.
The paper tackled the problem of harmful misinformation spreading across languages in multilingual LLMs, finding that standard unlearning methods focused on English are insufficient and can reinforce harm, and showing that addressing harmful responses in both English and the original language effectively eliminates generations for all languages.
This paper investigates the propagation of harmful information in multilingual large language models (LLMs) and evaluates the efficacy of various unlearning methods. We demonstrate that fake information, regardless of the language it is in, once introduced into these models through training data, can spread across different languages, compromising the integrity and reliability of the generated content. Our findings reveal that standard unlearning techniques, which typically focus on English data, are insufficient in mitigating the spread of harmful content in multilingual contexts and could inadvertently reinforce harmful content across languages. We show that only by addressing harmful responses in both English and the original language of the harmful data can we effectively eliminate generations for all languages. This underscores the critical need for comprehensive unlearning strategies that consider the multilingual nature of modern LLMs to enhance their safety and reliability across diverse linguistic landscapes.