FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields
It addresses 3D facial landmark estimation for face editing and swapping, representing a novel application but incremental in method.
This paper tackles the problem of directly predicting 3D facial landmarks from neural radiance fields (NeRFs), achieving accurate detection through a coarse-to-fine model with expression augmentation for training.
This paper presents the first significant work on directly predicting 3D face landmarks on neural radiance fields (NeRFs). Our 3D coarse-to-fine Face Landmarks NeRF (FLNeRF) model efficiently samples from a given face NeRF with individual facial features for accurate landmarks detection. Expression augmentation is applied to facial features in a fine scale to simulate large emotions range including exaggerated facial expressions (e.g., cheek blowing, wide opening mouth, eye blinking) for training FLNeRF. Qualitative and quantitative comparison with related state-of-the-art 3D facial landmark estimation methods demonstrate the efficacy of FLNeRF, which contributes to downstream tasks such as high-quality face editing and swapping with direct control using our NeRF landmarks. Code and data will be available. Github link: https://github.com/ZHANG1023/FLNeRF.