Improving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesis
This work addresses the problem of person-specific gaze adaptation for applications requiring accurate 3D gaze direction, but it is incremental as it builds on existing gaze redirection and domain adaptation methods.
The paper tackles the challenge of adapting gaze estimation models to individual users with limited training data by generating additional samples through gaze redirection synthesis, achieving improved performance on two public datasets.
As an indicator of human attention gaze is a subtle behavioral cue which can be exploited in many applications. However, inferring 3D gaze direction is challenging even for deep neural networks given the lack of large amount of data (groundtruthing gaze is expensive and existing datasets use different setups) and the inherent presence of gaze biases due to person-specific difference. In this work, we address the problem of person-specific gaze model adaptation from only a few reference training samples. The main and novel idea is to improve gaze adaptation by generating additional training samples through the synthesis of gaze-redirected eye images from existing reference samples. In doing so, our contributions are threefold: (i) we design our gaze redirection framework from synthetic data, allowing us to benefit from aligned training sample pairs to predict accurate inverse mapping fields; (ii) we proposed a self-supervised approach for domain adaptation; (iii) we exploit the gaze redirection to improve the performance of person-specific gaze estimation. Extensive experiments on two public datasets demonstrate the validity of our gaze retargeting and gaze estimation framework.