Fine-grained Identity Preserving Landmark Synthesis for Face Reenactment
This work addresses identity preservation in face reenactment for applications like video editing, but it appears incremental as it builds on existing landmark-based methods with refinements.
The paper tackles the problem of unsatisfactory identity preservation in face reenactment due to coarse reference landmarks by proposing a fine-grained landmark-guided approach, resulting in higher quality faces with fewer artifacts and clearer boundaries than state-of-the-art methods on BeautySelfie and VoxCeleb1 datasets.
Recent face reenactment works are limited by the coarse reference landmarks, leading to unsatisfactory identity preserving performance due to the distribution gap between the manipulated landmarks and those sampled from a real person. To address this issue, we propose a fine-grained identity-preserving landmark-guided face reenactment approach. The proposed method has two novelties. First, a landmark synthesis network which is designed to generate fine-grained landmark faces with more details. The network refines the manipulated landmarks and generates a smooth and gradually changing face landmark sequence with good identity preserving ability. Second, several novel loss functions including synthesized face identity preserving loss, foreground/background mask loss as well as boundary loss are designed, which aims at synthesizing clear and sharp high-quality faces. Experiments are conducted on our self-collected BeautySelfie and the public VoxCeleb1 datasets. The presented qualitative and quantitative results show that our method can reenact fine-grained higher quality faces with good ID-preserved appearance details, fewer artifacts and clearer boundaries than state-of-the-art works. Code will be released for reproduction.