UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization
This work addresses the high annotation costs and diverse demands in summarization by enabling shareable knowledge across tasks, though it is incremental as it builds on existing few-shot and pre-training paradigms.
The paper tackles the problem of few-shot summarization by proposing UniSumm, a unified model pre-trained on multiple summarization tasks, which outperforms strong baselines by a large margin across diverse tasks in the new SummZoo benchmark and achieves comparable results to GPT-3.5 in human evaluation.
The high annotation costs and diverse demands of various summarization tasks motivate the development of few-shot summarization. However, despite the emergence of many summarization tasks and datasets, the current training paradigm for few-shot summarization systems ignores potentially shareable knowledge in heterogeneous datasets. To this end, we propose \textsc{UniSumm}, a unified few-shot summarization model pre-trained with multiple summarization tasks and can be prefix-tuned to excel at any few-shot summarization task. Meanwhile, to better evaluate few-shot summarizers, under the principles of diversity and robustness, we assemble and release a new benchmark \textsc{SummZoo}. It consists of $8$ summarization tasks with multiple sets of few-shot samples for each task, covering diverse domains. Experimental results and analysis show that \textsc{UniSumm} outperforms strong baselines by a large margin across all sub-tasks in \textsc{SummZoo} under both automatic and human evaluations and achieves comparable results in human evaluation compared with a GPT-3.5 model.