GazePrior: Zero-Shot AR/VR Eye Tracking via Learned 3D Gaze Reconstruction
For AR/VR eye tracking developers, this work removes the need for device-specific data collection, enabling zero-shot generalization to new devices.
GazePrior introduces a data-driven 3D prior for human eyes to synthesize realistic training data for AR/VR eye tracking, eliminating the need for costly real data collection. The method outperforms previous zero-shot approaches, achieving higher accuracy and robustness.
Eye tracking (ET) is a foundational technology for advanced AR/VR applications. However, training ET models for every new ET device is challenging: real data collection is costly and time-consuming, while existing synthetic data generation methods lack realism. To remove the need for additional data collection while maintaining data quality, we introduce a data-driven 3D prior that models the distribution of human eyes across diverse identities, gaze directions, and light settings. This model, which we coin GazePrior, then enables sparse-input 3D reconstruction of annotated data collected with previous ET devices, which can in turn be rendered from the cameras of any target ET device. Our approach synthesizes data with the realism, diversity and ground-truth accuracy of real data collection without its prohibitive costs. Our experiments demonstrate that ET models trained with our synthesized data outperform previous zero-shot methods, achieving higher accuracy and robustness.