Personalized Saliency and its Prediction
This work addresses the need for personalized saliency prediction in computer vision, which is incremental as it builds on existing universal saliency models by adding a user-specific component.
The paper tackles the problem of predicting personalized visual saliency maps, which vary across observers, by decomposing them into universal and discrepancy components, and proposes two CNN-based models that achieve effective prediction and generalization to unseen observers.
Nearly all existing visual saliency models by far have focused on predicting a universal saliency map across all observers. Yet psychology studies suggest that visual attention of different observers can vary significantly under specific circumstances, especially a scene is composed of multiple salient objects. To study such heterogenous visual attention pattern across observers, we first construct a personalized saliency dataset and explore correlations between visual attention, personal preferences, and image contents. Specifically, we propose to decompose a personalized saliency map (referred to as PSM) into a universal saliency map (referred to as USM) predictable by existing saliency detection models and a new discrepancy map across users that characterizes personalized saliency. We then present two solutions towards predicting such discrepancy maps, i.e., a multi-task convolutional neural network (CNN) framework and an extended CNN with Person-specific Information Encoded Filters (CNN-PIEF). Extensive experimental results demonstrate the effectiveness of our models for PSM prediction as well their generalization capability for unseen observers.