VATEX Captioning Challenge 2019: Multi-modal Information Fusion and Multi-stage Training Strategy for Video Captioning
This work addresses video captioning for applications like accessibility or content indexing, but it is incremental as it builds on existing multi-modal fusion and training approaches.
The authors tackled video captioning by fusing multiple modalities (appearance, motion, audio) guided by video topic and using a multi-stage training strategy, achieving steady and significant improvements on the VATEX benchmark.
Multi-modal information is essential to describe what has happened in a video. In this work, we represent videos by various appearance, motion and audio information guided with video topic. By following multi-stage training strategy, our experiments show steady and significant improvement on the VATEX benchmark. This report presents an overview and comparative analysis of our system designed for both Chinese and English tracks on VATEX Captioning Challenge 2019.