Generating abstractive summaries of Lithuanian news articles using a transformer model
This work addresses the problem of generating summaries for Lithuanian news articles, which is incremental as it applies existing transformer methods to a new language domain.
The authors tackled abstractive summarization of Lithuanian news articles by training the first monolingual Lithuanian transformer model, achieving an average ROUGE-2 score of 0.163 with coherent summaries that sometimes contain misleading information.
In this work, we train the first monolingual Lithuanian transformer model on a relatively large corpus of Lithuanian news articles and compare various output decoding algorithms for abstractive news summarization. We achieve an average ROUGE-2 score 0.163, generated summaries are coherent and look impressive at first glance. However, some of them contain misleading information that is not so easy to spot. We describe all the technical details and share our trained model and accompanying code in an online open-source repository, as well as some characteristic samples of the generated summaries.