No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling
This addresses the problem of improving story generation quality for AI systems in visual storytelling, though it is incremental as it builds on existing methods with a novel reward learning approach.
The paper tackles the problem of generating abstract stories from photo streams, which is challenging due to expressive language and imaginary concepts not in images, and proposes an Adversarial Reward Learning (AREL) framework to learn rewards from human demonstrations. The result shows slight performance boost in automatic evaluation but significant improvement in human evaluation for generating more human-like stories compared to state-of-the-art methods.
Though impressive results have been achieved in visual captioning, the task of generating abstract stories from photo streams is still a little-tapped problem. Different from captions, stories have more expressive language styles and contain many imaginary concepts that do not appear in the images. Thus it poses challenges to behavioral cloning algorithms. Furthermore, due to the limitations of automatic metrics on evaluating story quality, reinforcement learning methods with hand-crafted rewards also face difficulties in gaining an overall performance boost. Therefore, we propose an Adversarial REward Learning (AREL) framework to learn an implicit reward function from human demonstrations, and then optimize policy search with the learned reward function. Though automatic eval- uation indicates slight performance boost over state-of-the-art (SOTA) methods in cloning expert behaviors, human evaluation shows that our approach achieves significant improvement in generating more human-like stories than SOTA systems.