Real Eyes Realize Faster: Gaze Stability and Pupil Novelty for Efficient Egocentric Learning
This work offers a method for efficient, always-on egocentric data curation for embodied robotics, imitation learning, and assistive AR, which is crucial given the storage and battery constraints of wearable devices.
This paper addresses the problem of redundant and low-quality frames in egocentric video streams by proposing a Dual-Criterion Frame Curator that uses gaze stability and pupil response from eye-tracking headsets. The method achieves classification performance matching the full stream with only 10% of the frames on the Visual Experience Dataset (VEDB).
Always-on egocentric cameras are increasingly used as demonstrations for embodied robotics, imitation learning, and assistive AR, but the resulting video streams are dominated by redundant and low-quality frames. Under the storage and battery constraints of wearable devices, choosing which frames to keep is as important as how to learn from them. We observe that modern eye-tracking headsets provide a continuous, training-free side channel that decomposes into two complementary axes: gaze fixation captures visual stability (quality), while pupil response captures arousal-linked moments (novelty). We operationalize this insight as a Dual-Criterion Frame Curator that first gates frames by gaze quality and then ranks the survivors by pupil-derived novelty. On the Visual Experience Dataset (VEDB), curated frames at 10% budget match the classification performance of the full stream, and naive signal fusion consistently destroys both contributions. The benefit is task-dependent: pupil ranking improves activity recognition, while gaze-only selection already dominates for scene recognition, confirming that the two signals serve genuinely different roles. Our method requires no model inference and operates at capture time, offering a path toward efficient, always-on egocentric data curation.