Learning to Attend Relevant Regions in Videos from Eye Fixations
This work addresses video analysis for applications like entertainment and robotic control, but it is incremental as it builds on existing attention models with eye fixation data.
The paper tackled the problem of identifying relevant semantic regions in videos by using an RNN-based visual attention model trained on eye fixation labels, finding that the approach shows good potential but performance depends heavily on label quality.
Attentively important regions in video frames account for a majority part of the semantics in each frame. This information is helpful in many applications not only for entertainment (such as auto generating commentary and tourist guide) but also for robotic control which holds a larascope supported for laparoscopic surgery. However, it is not always straightforward to define and locate such semantic regions in videos. In this work, we attempt to address the problem of attending relevant regions in videos by leveraging the eye fixations labels with a RNN-based visual attention model. Our experimental results suggest that this approach holds a good potential to learn to attend semantic regions in videos while its performance also heavily relies on the quality of eye fixations labels.