HERA: Improving Long Document Summarization using Large Language Models with Context Packaging and Reordering
This addresses the challenge of scattered information and messy narrative order in long documents for summarization tasks, though it is incremental as it builds on existing LLM capabilities.
The paper tackles the problem of poor performance of large language models (LLMs) in long document summarization by proposing HERA, a framework that segments documents, retrieves event-related segments, and reorders them, resulting in improved ROUGE, BERTScore, and faithfulness metrics without fine-tuning.
Despite the rapid growth of context length of large language models (LLMs) , LLMs still perform poorly in long document summarization. An important reason for this is that relevant information about an event is scattered throughout long documents, and the messy narrative order impairs the accurate understanding and utilization of LLMs for long documents. To address these issues, we propose a novel summary generation framework, called HERA. Specifically, we first segment a long document by its semantic structure and retrieve text segments about the same event, and finally reorder them to form the input context. We evaluate our approach on two long document summarization datasets. The experimental results show that HERA outperforms foundation models in ROUGE, BERTScore and faithfulness metrics, while HERA does not require additional fine-tuning and resources.