The Forgettable-Watcher Model for Video Question Answering
This addresses video-QA, a less explored area compared to image-QA, with incremental improvements in handling temporal and phrasal structures.
The authors tackled video question answering by proposing the forgettable-watcher model with re-watching and re-reading mechanisms, and introduced the TGIF-QA dataset, showing effectiveness in experiments.
A number of visual question answering approaches have been proposed recently, aiming at understanding the visual scenes by answering the natural language questions. While the image question answering has drawn significant attention, video question answering is largely unexplored. Video-QA is different from Image-QA since the information and the events are scattered among multiple frames. In order to better utilize the temporal structure of the videos and the phrasal structures of the answers, we propose two mechanisms: the re-watching and the re-reading mechanisms and combine them into the forgettable-watcher model. Then we propose a TGIF-QA dataset for video question answering with the help of automatic question generation. Finally, we evaluate the models on our dataset. The experimental results show the effectiveness of our proposed models.