AmbientEye: A Dataset for Pupil Segmentation under Natural Ambient Infrared Illumination
For researchers developing eye-tracking for smart glasses, this dataset provides the first benchmark for pupil segmentation under natural ambient light, highlighting a significant performance gap.
The paper introduces AmbientEye, a large-scale dataset of 2.6 million eye images captured outdoors under natural sunlight without active IR illumination, and benchmarks a state-of-the-art pupil segmentation algorithm, finding a performance drop from 0.928 to 0.767 compared to controlled IR datasets.
Eye tracking is essential for smart glasses, as it provides insight into user attention for ambient intelligence applications. However, most existing eye-tracking systems rely on active infrared (IR) illumination, creating practical barriers to all-day outdoor use due to power consumption. In this paper, we investigate whether passive IR cameras alone, without any active IR light source, can enable reliable pupil detection in unconstrained outdoor environments, where ambient sunlight serves as the sole illumination source. To support this investigation, we introduce AmbientEye, a large-scale dataset of 2,606,225 eye images collected from 35 participants from 19 countries. It is captured outdoors under natural sunlight with two off-axis camera configurations and two sun-orientation conditions. We provide high-quality pupil annotation through SAM2 automatic segmentation, followed by refinement by human annotators. We benchmark a state-of-the-art pupil segmentation algorithm on our dataset and compare its performance with that on existing datasets under controlled IR illumination. Results reveal a substantial drop in pupil segmentation performance from 0.928 on controlled IR datasets to 0.767 on AmbientEye. This performance gap highlights the challenge of the ambient-light setting. This positions AmbientEye as a first benchmark for an unexplored and highly practical eye-tracking scenario.