Sparse to Dense Motion Transfer for Face Image Animation
This addresses the challenge of face animation with limited input data, which is incremental as it builds on existing methods but improves cross-identity performance.
The paper tackles the problem of animating a face image using only sparse landmarks as the driving signal, achieving results comparable to state-of-the-art methods on same-identity testing and better results on cross-identity testing.
Face image animation from a single image has achieved remarkable progress. However, it remains challenging when only sparse landmarks are available as the driving signal. Given a source face image and a sequence of sparse face landmarks, our goal is to generate a video of the face imitating the motion of landmarks. We develop an efficient and effective method for motion transfer from sparse landmarks to the face image. We then combine global and local motion estimation in a unified model to faithfully transfer the motion. The model can learn to segment the moving foreground from the background and generate not only global motion, such as rotation and translation of the face, but also subtle local motion such as the gaze change. We further improve face landmark detection on videos. With temporally better aligned landmark sequences for training, our method can generate temporally coherent videos with higher visual quality. Experiments suggest we achieve results comparable to the state-of-the-art image driven method on the same identity testing and better results on cross identity testing.