Cross-lingual Transfer of Abstractive Summarizer to Less-resource Language
This work addresses the challenge of abstractive summarization for less-resourced languages, specifically Slovene, by leveraging cross-lingual transfer, which is an incremental step towards broader language support.
This paper explores cross-lingual transfer of an abstractive summarizer from English to Slovene, a less-resourced language. By using an additional language model for target language evaluation, the authors achieved summaries with quality similar to a Slovene-only trained model, as measured by automatic evaluation, and high accuracy with acceptable readability in human evaluation.
Automatic text summarization extracts important information from texts and presents the information in the form of a summary. Abstractive summarization approaches progressed significantly by switching to deep neural networks, but results are not yet satisfactory, especially for languages where large training sets do not exist. In several natural language processing tasks, a cross-lingual model transfer is successfully applied in less-resource languages. For summarization, the cross-lingual model transfer was not attempted due to a non-reusable decoder side of neural models that cannot correct target language generation. In our work, we use a pre-trained English summarization model based on deep neural networks and sequence-to-sequence architecture to summarize Slovene news articles. We address the problem of inadequate decoder by using an additional language model for the evaluation of the generated text in target language. We test several cross-lingual summarization models with different amounts of target data for fine-tuning. We assess the models with automatic evaluation measures and conduct a small-scale human evaluation. Automatic evaluation shows that the summaries of our best cross-lingual model are useful and of quality similar to the model trained only in the target language. Human evaluation shows that our best model generates summaries with high accuracy and acceptable readability. However, similar to other abstractive models, our models are not perfect and may occasionally produce misleading or absurd content.