Self-Attentive Model for Headline Generation
This addresses headline generation for news summarization, presenting incremental improvements with new benchmark results.
The paper tackled the problem of headline generation for news articles by applying a Universal Transformer with byte-pair encoding, achieving state-of-the-art results with ROUGE-L F1-scores of 24.84 on the New York Times corpus and 36.81 on a new RIA corpus.
Headline generation is a special type of text summarization task. While the amount of available training data for this task is almost unlimited, it still remains challenging, as learning to generate headlines for news articles implies that the model has strong reasoning about natural language. To overcome this issue, we applied recent Universal Transformer architecture paired with byte-pair encoding technique and achieved new state-of-the-art results on the New York Times Annotated corpus with ROUGE-L F1-score 24.84 and ROUGE-2 F1-score 13.48. We also present the new RIA corpus and reach ROUGE-L F1-score 36.81 and ROUGE-2 F1-score 22.15 on it.