ATSal: An Attention Based Architecture for Saliency Prediction in 360 Videos
This work is significant for developers of head-mounted display systems, as it improves the efficiency of rendering 360-degree videos by predicting user attention.
This paper addresses the challenge of saliency prediction in 360-degree videos by proposing ATSal, an attention-based model. The model explicitly encodes global static visual attention, enabling expert models to focus on local patch saliency across frames, and it outperforms existing state-of-the-art methods on two datasets.
The spherical domain representation of 360 video/image presents many challenges related to the storage, processing, transmission and rendering of omnidirectional videos (ODV). Models of human visual attention can be used so that only a single viewport is rendered at a time, which is important when developing systems that allow users to explore ODV with head mounted displays (HMD). Accordingly, researchers have proposed various saliency models for 360 video/images. This paper proposes ATSal, a novel attention based (head-eye) saliency model for 360\degree videos. The attention mechanism explicitly encodes global static visual attention allowing expert models to focus on learning the saliency on local patches throughout consecutive frames. We compare the proposed approach to other state-of-the-art saliency models on two datasets: Salient360! and VR-EyeTracking. Experimental results on over 80 ODV videos (75K+ frames) show that the proposed method outperforms the existing state-of-the-art.