MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targets
This addresses the problem of realistic face reenactment for unseen identities in few-shot settings, which is incremental as it builds on existing reenactment methods by specifically targeting identity preservation.
The paper tackled the identity preservation problem in few-shot face reenactment, where mismatches between target and driver identities degrade output quality, and introduced MarioNETte with components like an image attention block and landmark transformer to produce high-quality reenactments of unseen identities, outperforming all baselines.
When there is a mismatch between the target identity and the driver identity, face reenactment suffers severe degradation in the quality of the result, especially in a few-shot setting. The identity preservation problem, where the model loses the detailed information of the target leading to a defective output, is the most common failure mode. The problem has several potential sources such as the identity of the driver leaking due to the identity mismatch, or dealing with unseen large poses. To overcome such problems, we introduce components that address the mentioned problem: image attention block, target feature alignment, and landmark transformer. Through attending and warping the relevant features, the proposed architecture, called MarioNETte, produces high-quality reenactments of unseen identities in a few-shot setting. In addition, the landmark transformer dramatically alleviates the identity preservation problem by isolating the expression geometry through landmark disentanglement. Comprehensive experiments are performed to verify that the proposed framework can generate highly realistic faces, outperforming all other baselines, even under a significant mismatch of facial characteristics between the target and the driver.