GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting
This work addresses generalization challenges in gaze estimation for computer vision applications, representing an incremental improvement by adapting 3D Gaussian Splatting to a specific domain.
The paper tackles the problem of gaze redirection for gaze estimation by proposing GazeGaussian, a method that uses a two-stream 3D Gaussian Splatting model to achieve high-fidelity results, outperforming existing methods in rendering speed, gaze redirection accuracy, and facial synthesis across multiple datasets.
Gaze estimation encounters generalization challenges when dealing with out-of-distribution data. To address this problem, recent methods use neural radiance fields (NeRF) to generate augmented data. However, existing methods based on NeRF are computationally expensive and lack facial details. 3D Gaussian Splatting (3DGS) has become the prevailing representation of neural fields. While 3DGS has been extensively examined in head avatars, it faces challenges with accurate gaze control and generalization across different subjects. In this work, we propose GazeGaussian, the first high-fidelity gaze redirection method that uses a two-stream 3DGS model to represent the face and eye regions separately. Leveraging the unstructured nature of 3DGS, we develop a novel representation of the eye for rigid eye rotation based on the target gaze direction. To enable synthesis generalization across various subjects, we integrate an expression-guided module to inject subject-specific information into the neural renderer. Comprehensive experiments show that GazeGaussian outperforms existing methods in rendering speed, gaze redirection accuracy, and facial synthesis across multiple datasets. The code is available at: https://ucwxb.github.io/GazeGaussian.