Egocentric Field-of-View Localization Using First-Person Point-of-View Devices
This addresses the need for precise localization of visual attention in mixed reality and behavioral analysis, though it appears incremental as it builds on existing matching and sensor fusion techniques.
The paper tackles the problem of determining what a person is attending to by matching first-person visual data with a reference corpus to perform egocentric field-of-view localization, achieving applications in augmented reality and social interaction studies.
We present a technique that uses images, videos and sensor data taken from first-person point-of-view devices to perform egocentric field-of-view (FOV) localization. We define egocentric FOV localization as capturing the visual information from a person's field-of-view in a given environment and transferring this information onto a reference corpus of images and videos of the same space, hence determining what a person is attending to. Our method matches images and video taken from the first-person perspective with the reference corpus and refines the results using the first-person's head orientation information obtained using the device sensors. We demonstrate single and multi-user egocentric FOV localization in different indoor and outdoor environments with applications in augmented reality, event understanding and studying social interactions.