Large Language Models for Summarizing Czech Historical Documents and Beyond
This work addresses the underexplored area of Czech historical document summarization, which is incremental by applying existing models to a new domain and dataset.
The paper tackled the problem of Czech text summarization, particularly for historical documents, by employing large language models like Mistral and mT5, achieving new state-of-the-art results on the modern Czech dataset SumeCzech and introducing a novel dataset for historical documents with baseline results.
Text summarization is the task of shortening a larger body of text into a concise version while retaining its essential meaning and key information. While summarization has been significantly explored in English and other high-resource languages, Czech text summarization, particularly for historical documents, remains underexplored due to linguistic complexities and a scarcity of annotated datasets. Large language models such as Mistral and mT5 have demonstrated excellent results on many natural language processing tasks and languages. Therefore, we employ these models for Czech summarization, resulting in two key contributions: (1) achieving new state-of-the-art results on the modern Czech summarization dataset SumeCzech using these advanced models, and (2) introducing a novel dataset called Posel od Čerchova for summarization of historical Czech documents with baseline results. Together, these contributions provide a great potential for advancing Czech text summarization and open new avenues for research in Czech historical text processing.