Visual Attention on the Sun: What Do Existing Models Actually Predict?
This work addresses a critical concern for the computer vision community by revealing potential limitations in deep learning models for visual attention prediction, offering a complementary perspective through a novel dataset.
The paper tackles the problem of whether existing deep visual attention models truly capture human attention mechanisms by introducing a new dataset, VASUN, with irregular solar images. It finds that state-of-the-art deep models perform poorly on this dataset, while classic shallow models perform well, suggesting deep models may rely on memorizing specific patterns rather than learning inherent mechanisms.
Visual attention prediction is a classic problem that seems to be well addressed in the deep learning era. One compelling concern, however, gradually arise along with the rapidly growing performance scores over existing visual attention datasets: do existing deep models really capture the inherent mechanism of human visual attention? To address this concern, this paper proposes a new dataset, named VASUN, that records the free-viewing human attention on solar images. Different from previous datasets, images in VASUN contain many irregular visual patterns that existing deep models have never seen. By benchmarking existing models on VASUN, we find the performances of many state-of-the-art deep models drop remarkably, while many classic shallow models perform impressively. From these results, we find that the significant performance advance of existing deep attention models may come from their capabilities of memorizing and predicting the occurrence of some specific visual patterns other than learning the inherent mechanism of human visual attention. In addition, we also train several baseline models on VASUN to demonstrate the feasibility and key issues of predicting visual attention on the sun. These baseline models, together with the proposed dataset, can be used to revisit the problem of visual attention prediction from a novel perspective that are complementary to existing ones.