A study of parameters affecting visual saliency assessment
This work addresses the problem of model evaluation for researchers in computer vision, but it is incremental as it builds on existing evaluation methods without introducing a new model.
The paper tackles the challenge of fairly evaluating computational visual saliency models by proposing a new framework based on three experiments that analyze differences between ground truths, properties of salient regions, and metrics, using statistical analysis to address these questions.
Since the early 2000s, computational visual saliency has been a very active research area. Each year, more and more new models are published in the main computer vision conferences. Nowadays, one of the big challenges is to find a way to fairly evaluate all of these models. In this paper, a new framework is proposed to assess models of visual saliency. This evaluation is divided into three experiments leading to the proposition of a new evaluation framework. Each experiment is based on a basic question: 1) there are two ground truths for saliency evaluation: what are the differences between eye fixations and manually segmented salient regions?, 2) the properties of the salient regions: for example, do large, medium and small salient regions present different difficulties for saliency models? and 3) the metrics used to assess saliency models: what advantages would there be to mix them with PCA? Statistical analysis is used here to answer each of these three questions.