Multi-Task Learning for Cross-Lingual Abstractive Summarization
This work addresses data scarcity in cross-lingual summarization for NLP researchers, offering incremental improvements by integrating existing data sources.
The paper tackles cross-lingual abstractive summarization by proposing a multi-task learning framework called Transum that incorporates genuine data like translation pairs and monolingual summaries, achieving top ROUGE scores on Chinese-English and Arabic-English summarization and improving performance over strong baselines in machine translation tasks.
We present a multi-task learning framework for cross-lingual abstractive summarization to augment training data. Recent studies constructed pseudo cross-lingual abstractive summarization data to train their neural encoder-decoders. Meanwhile, we introduce existing genuine data such as translation pairs and monolingual abstractive summarization data into training. Our proposed method, Transum, attaches a special token to the beginning of the input sentence to indicate the target task. The special token enables us to incorporate the genuine data into the training data easily. The experimental results show that Transum achieves better performance than the model trained with only pseudo cross-lingual summarization data. In addition, we achieve the top ROUGE score on Chinese-English and Arabic-English abstractive summarization. Moreover, Transum also has a positive effect on machine translation. Experimental results indicate that Transum improves the performance from the strong baseline, Transformer, in Chinese-English, Arabic-English, and English-Japanese translation datasets.