Low-Resource Neural Headline Generation
This addresses the challenge of generating high-quality headlines with limited data, which is incremental as it builds on existing pretraining methods for a specific domain.
The paper tackled the problem of neural headline generation in low-resource settings by improving headline quality on smaller datasets through pretraining, achieving up to 32.4% relative improvement in perplexity and 2.84 points in ROUGE.
Recent neural headline generation models have shown great results, but are generally trained on very large datasets. We focus our efforts on improving headline quality on smaller datasets by the means of pretraining. We propose new methods that enable pre-training all the parameters of the model and utilize all available text, resulting in improvements by up to 32.4% relative in perplexity and 2.84 points in ROUGE.