Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction
This work addresses the need for more accurate and variable human gaze prediction, which is crucial for applications in human-computer interaction, autonomous systems, and cognitive robotics, representing a strong incremental advance in the field.
The paper tackles the problem of predicting human gaze scanpaths by developing ScanDiff, a novel architecture that combines diffusion models with Vision Transformers to generate diverse and realistic gaze trajectories, surpassing state-of-the-art methods in both free-viewing and task-driven scenarios.
Predicting human gaze scanpaths is crucial for understanding visual attention, with applications in human-computer interaction, autonomous systems, and cognitive robotics. While deep learning models have advanced scanpath prediction, most existing approaches generate averaged behaviors, failing to capture the variability of human visual exploration. In this work, we present ScanDiff, a novel architecture that combines diffusion models with Vision Transformers to generate diverse and realistic scanpaths. Our method explicitly models scanpath variability by leveraging the stochastic nature of diffusion models, producing a wide range of plausible gaze trajectories. Additionally, we introduce textual conditioning to enable task-driven scanpath generation, allowing the model to adapt to different visual search objectives. Experiments on benchmark datasets show that ScanDiff surpasses state-of-the-art methods in both free-viewing and task-driven scenarios, producing more diverse and accurate scanpaths. These results highlight its ability to better capture the complexity of human visual behavior, pushing forward gaze prediction research. Source code and models are publicly available at https://aimagelab.github.io/ScanDiff.