Learning Human Visual Attention on 3D Surfaces through Geometry-Queried Semantic Priors
This work solves the problem of accurately predicting human visual attention on 3D surfaces for applications in computer vision and human-computer interaction, representing a novel method for a known bottleneck.
The paper tackled the problem of modeling human visual attention on 3D objects by addressing the gap in existing methods that lack semantic awareness, introducing SemGeo-AttentionNet, which achieved substantial improvements on datasets like SAL3D, NUS3D, and 3DVA.
Human visual attention on three-dimensional objects emerges from the interplay between bottom-up geometric processing and top-down semantic recognition. Existing 3D saliency methods rely on hand-crafted geometric features or learning-based approaches that lack semantic awareness, failing to explain why humans fixate on semantically meaningful but geometrically unremarkable regions. We introduce SemGeo-AttentionNet, a dual-stream architecture that explicitly formalizes this dichotomy through asymmetric cross-modal fusion, leveraging diffusion-based semantic priors from geometry-conditioned multi-view rendering and point cloud transformers for geometric processing. Cross-attention ensures geometric features query semantic content, enabling bottom-up distinctiveness to guide top-down retrieval. We extend our framework to temporal scanpath generation through reinforcement learning, introducing the first formulation respecting 3D mesh topology with inhibition-of-return dynamics. Evaluation on SAL3D, NUS3D and 3DVA datasets demonstrates substantial improvements, validating how cognitively motivated architectures effectively model human visual attention on three-dimensional surfaces.