SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency Volumes
This addresses the problem of predicting human visual attention in immersive environments like virtual reality, but it is incremental as it builds on existing saliency methods for 360-degree images.
The paper tackles scan-path prediction on 360-degree images by introducing SaltiNet, a deep neural network that uses a novel saliency volume representation, resulting in improved performance as shown in experiments.
We introduce SaltiNet, a deep neural network for scanpath prediction trained on 360-degree images. The model is based on a temporal-aware novel representation of saliency information named the saliency volume. The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes. Sampling strategies over these volumes are used to generate scanpaths over the 360-degree images. Our experiments show the advantages of using saliency volumes, and how they can be used for related tasks. Our source code and trained models available at https://github.com/massens/saliency-360salient-2017.