Vatex Video Captioning Challenge 2020: Multi-View Features and Hybrid Reward Strategies for Video Captioning
This work addresses video captioning for multilingual applications, but it is incremental as it builds on their previous VATEX 2019 method.
The authors tackled the VATEX Video Captioning Challenge 2020 by generating video descriptions in English and Chinese, achieving competitive results through multi-view features, hybrid reward strategies, and ensemble methods.
This report describes our solution for the VATEX Captioning Challenge 2020, which requires generating descriptions for the videos in both English and Chinese languages. We identified three crucial factors that improve the performance, namely: multi-view features, hybrid reward, and diverse ensemble. Based on our method of VATEX 2019 challenge, we achieved significant improvements this year with more advanced model architectures, combination of appearance and motion features, and careful hyper-parameters tuning. Our method achieves very competitive results on both of the Chinese and English video captioning tracks.