State-of-the-Art in Human Scanpath Prediction
This work addresses a benchmarking gap for researchers in computer vision and cognitive science, offering a standardized evaluation framework for scanpath prediction models.
The paper tackles the lack of principled comparison in human scanpath prediction models by evaluating them based on predicting each fixation given previous history, aligning with biological processes and using metrics like AUC and NSS. It provides a detailed state-of-the-art analysis on datasets MIT1003, MIT300, CAT2000 train, and CAT2000 test, showing the method allows for insights into model failures.
The last years have seen a surge in models predicting the scanpaths of fixations made by humans when viewing images. However, the field is lacking a principled comparison of those models with respect to their predictive power. In the past, models have usually been evaluated based on comparing human scanpaths to scanpaths generated from the model. Here, instead we evaluate models based on how well they predict each fixation in a scanpath given the previous scanpath history. This makes model evaluation closely aligned with the biological processes thought to underly scanpath generation and allows to apply established saliency metrics like AUC and NSS in an intuitive and interpretable way. We evaluate many existing models of scanpath prediction on the datasets MIT1003, MIT300, CAT2000 train and CAT200 test, for the first time giving a detailed picture of the current state of the art of human scanpath prediction. We also show that the discussed method of model benchmarking allows for more detailed analyses leading to interesting insights about where and when models fail to predict human behaviour. The MIT/Tuebingen Saliency Benchmark will implement the evaluation of scanpath models as detailed here, allowing researchers to score their models on the established benchmark datasets MIT300 and CAT2000.