4D Attention: Comprehensive Framework for Spatio-Temporal Gaze Mapping
This addresses the problem of human attention mapping for applications like human perceptual analysis or human-robot interaction, though it appears incremental as it builds on existing localization and reconstruction techniques.
The study tackled the challenge of measuring human attention in dynamic environments by developing the 4D Attention framework for unified gaze mapping onto static and dynamic objects, with quantitative evaluations showing its effectiveness.
This study presents a framework for capturing human attention in the spatio-temporal domain using eye-tracking glasses. Attention mapping is a key technology for human perceptual activity analysis or Human-Robot Interaction (HRI) to support human visual cognition; however, measuring human attention in dynamic environments is challenging owing to the difficulty in localizing the subject and dealing with moving objects. To address this, we present a comprehensive framework, 4D Attention, for unified gaze mapping onto static and dynamic objects. Specifically, we estimate the glasses pose by leveraging a loose coupling of direct visual localization and Inertial Measurement Unit (IMU) values. Further, by installing reconstruction components into our framework, dynamic objects not captured in the 3D environment map are instantiated based on the input images. Finally, a scene rendering component synthesizes a first-person view with identification (ID) textures and performs direct 2D-3D gaze association. Quantitative evaluations showed the effectiveness of our framework. Additionally, we demonstrated the applications of 4D Attention through experiments in real situations.