Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization
This work addresses the problem of optimizing multilingual fine-tuning for summarization, providing practical insights for researchers and practitioners, though it is incremental in nature.
The paper conducted an extensive empirical evaluation of multilingual fine-tuning methods for text summarization across 45 languages, establishing a new state-of-the-art on the XL-Sum dataset and deriving observations to guide future research.
Recently, a large number of tuning strategies have been proposed to adapt pre-trained language models to downstream tasks. In this paper, we perform an extensive empirical evaluation of various tuning strategies for multilingual learning, particularly in the context of text summarization. Specifically, we explore the relative advantages of three families of multilingual tuning strategies (a total of five models) and empirically evaluate them for summarization over 45 languages. Experimentally, we not only established a new state-of-the-art on the XL-Sum dataset but also derive a series of observations that hopefully can provide hints for future research on the design of multilingual tuning strategies.