DocSum: Domain-Adaptive Pre-training for Document Abstractive Summarization
This work addresses the challenge of summarizing administrative documents for business and organizational settings, though it appears incremental as it adapts existing methods to a specific domain.
The paper tackled the problem of abstractive summarization for administrative documents, which face domain-specific terminology and OCR errors, by introducing DocSum, a domain-adaptive framework that uses pre-training on OCR-transcribed text and fine-tuning with question-answer pairs, resulting in improved summary accuracy and relevance as demonstrated in experiments.
Abstractive summarization has made significant strides in condensing and rephrasing large volumes of text into coherent summaries. However, summarizing administrative documents presents unique challenges due to domain-specific terminology, OCR-generated errors, and the scarcity of annotated datasets for model fine-tuning. Existing models often struggle to adapt to the intricate structure and specialized content of such documents. To address these limitations, we introduce DocSum, a domain-adaptive abstractive summarization framework tailored for administrative documents. Leveraging pre-training on OCR-transcribed text and fine-tuning with an innovative integration of question-answer pairs, DocSum enhances summary accuracy and relevance. This approach tackles the complexities inherent in administrative content, ensuring outputs that align with real-world business needs. To evaluate its capabilities, we define a novel downstream task setting-Document Abstractive Summarization-which reflects the practical requirements of business and organizational settings. Comprehensive experiments demonstrate DocSum's effectiveness in producing high-quality summaries, showcasing its potential to improve decision-making and operational workflows across the public and private sectors.