Reinforced Video Captioning with Entailment Rewards
This work addresses the challenge of generating accurate and logically consistent captions for videos, which is important for applications in accessibility and content analysis, representing an incremental improvement over existing methods.
The paper tackles the problem of video captioning by optimizing sentence-level metrics with reinforcement learning, achieving significant improvements over baselines on multiple datasets, and further enhances performance with a novel entailment-based reward (CIDEnt) that corrects for contradictions, setting a new state-of-the-art on the MSR-VTT dataset.
Sequence-to-sequence models have shown promising improvements on the temporal task of video captioning, but they optimize word-level cross-entropy loss during training. First, using policy gradient and mixed-loss methods for reinforcement learning, we directly optimize sentence-level task-based metrics (as rewards), achieving significant improvements over the baseline, based on both automatic metrics and human evaluation on multiple datasets. Next, we propose a novel entailment-enhanced reward (CIDEnt) that corrects phrase-matching based metrics (such as CIDEr) to only allow for logically-implied partial matches and avoid contradictions, achieving further significant improvements over the CIDEr-reward model. Overall, our CIDEnt-reward model achieves the new state-of-the-art on the MSR-VTT dataset.