NumHG: A Dataset for Number-Focused Headline Generation
This addresses a specific issue in abstractive summarization for news applications, but it is incremental as it focuses on dataset creation and benchmarking rather than a novel method.
The authors tackled the problem of inaccurate numeral generation in headline generation by introducing the NumHG dataset with over 27,000 annotated numeral-rich news articles, and their evaluation of five models showed a need for improvement in numerical accuracy.
Headline generation, a key task in abstractive summarization, strives to condense a full-length article into a succinct, single line of text. Notably, while contemporary encoder-decoder models excel based on the ROUGE metric, they often falter when it comes to the precise generation of numerals in headlines. We identify the lack of datasets providing fine-grained annotations for accurate numeral generation as a major roadblock. To address this, we introduce a new dataset, the NumHG, and provide over 27,000 annotated numeral-rich news articles for detailed investigation. Further, we evaluate five well-performing models from previous headline generation tasks using human evaluation in terms of numerical accuracy, reasonableness, and readability. Our study reveals a need for improvement in numerical accuracy, demonstrating the potential of the NumHG dataset to drive progress in number-focused headline generation and stimulate further discussions in numeral-focused text generation.