AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarization
This addresses the understudied problem of Arabic abstractive summarization for NLP researchers and practitioners, though it is incremental as it adapts an existing method to a new language.
The authors tackled the lack of pretrained sequence-to-sequence models for Arabic abstractive summarization by proposing AraBART, which achieved the best performance on multiple datasets, outperforming strong baselines like Arabic BERT-based models and multilingual mBART and mT5.
Like most natural language understanding and generation tasks, state-of-the-art models for summarization are transformer-based sequence-to-sequence architectures that are pretrained on large corpora. While most existing models focused on English, Arabic remained understudied. In this paper we propose AraBART, the first Arabic model in which the encoder and the decoder are pretrained end-to-end, based on BART. We show that AraBART achieves the best performance on multiple abstractive summarization datasets, outperforming strong baselines including a pretrained Arabic BERT-based model and multilingual mBART and mT5 models.