MagicEyes: A Large Scale Eye Gaze Estimation Dataset for Mixed Reality
This addresses the problem of accurate eye tracking for mixed reality applications, such as energy-efficient rendering and interaction, but it is incremental as it primarily provides a new dataset and model for an existing domain.
The authors tackled the challenge of eye gaze estimation in mixed reality by introducing MagicEyes, the first large-scale dataset collected from real MR devices, which includes 587 subjects and over 800,000 images with gaze labels, and they proposed a multi-task EyeNet model that achieved competitive performance on this new benchmark.
With the emergence of Virtual and Mixed Reality (XR) devices, eye tracking has received significant attention in the computer vision community. Eye gaze estimation is a crucial component in XR -- enabling energy efficient rendering, multi-focal displays, and effective interaction with content. In head-mounted XR devices, the eyes are imaged off-axis to avoid blocking the field of view. This leads to increased challenges in inferring eye related quantities and simultaneously provides an opportunity to develop accurate and robust learning based approaches. To this end, we present MagicEyes, the first large scale eye dataset collected using real MR devices with comprehensive ground truth labeling. MagicEyes includes $587$ subjects with $80,000$ images of human-labeled ground truth and over $800,000$ images with gaze target labels. We evaluate several state-of-the-art methods on MagicEyes and also propose a new multi-task EyeNet model designed for detecting the cornea, glints and pupil along with eye segmentation in a single forward pass.