Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning
This work addresses the challenge of creating attention-grabbing headlines for media and content creators, though it is incremental as it builds on existing reinforcement learning and headline generation techniques.
The paper tackled the problem of generating sensational headlines without labeled data by using a sensationalism scorer trained on clickbait headlines and a novel Auto-tuned Reinforcement Learning (ARL) method to balance reinforcement learning with maximum likelihood estimation. The result showed that 60.8% of samples generated by the model were rated as sensational, significantly outperforming baselines like Pointer-Gen and other RL models.
Sensational headlines are headlines that capture people's attention and generate reader interest. Conventional abstractive headline generation methods, unlike human writers, do not optimize for maximal reader attention. In this paper, we propose a model that generates sensational headlines without labeled data. We first train a sensationalism scorer by classifying online headlines with many comments ("clickbait") against a baseline of headlines generated from a summarization model. The score from the sensationalism scorer is used as the reward for a reinforcement learner. However, maximizing the noisy sensationalism reward will generate unnatural phrases instead of sensational headlines. To effectively leverage this noisy reward, we propose a novel loss function, Auto-tuned Reinforcement Learning (ARL), to dynamically balance reinforcement learning (RL) with maximum likelihood estimation (MLE). Human evaluation shows that 60.8% of samples generated by our model are sensational, which is significantly better than the Pointer-Gen baseline and other RL models.