Key-Element-Informed sLLM Tuning for Document Summarization
This work addresses the accessibility and cost issues in document summarization for users relying on smaller models, though it is incremental as it builds on existing instruction tuning approaches.
The paper tackles the problem of small-scale LLMs (sLLMs) missing key information in long document summarization by proposing KEITSum, a key-element-informed instruction tuning method, which improves relevance and reduces hallucinations, making sLLMs competitive with proprietary LLMs on dialogue and news datasets.
Remarkable advances in large language models (LLMs) have enabled high-quality text summarization. However, this capability is currently accessible only through LLMs of substantial size or proprietary LLMs with usage fees. In response, smaller-scale LLMs (sLLMs) of easy accessibility and low costs have been extensively studied, yet they often suffer from missing key information and entities, i.e., low relevance, in particular, when input documents are long. We hence propose a key-element-informed instruction tuning for summarization, so-called KEITSum, which identifies key elements in documents and instructs sLLM to generate summaries capturing these key elements. Experimental results on dialogue and news datasets demonstrate that sLLM with KEITSum indeed provides high-quality summarization with higher relevance and less hallucinations, competitive to proprietary LLM.