Automatic Face Reenactment
This enables automatic facial reenactment for video editing or entertainment applications without requiring extensive data or similar performances, though it is incremental as it builds on existing image retrieval and warping techniques.
The authors tackled the problem of automatically replacing an actor's face in a target video with a user's face from a source video while preserving the original performance, achieving convincing results using only a short source video captured with an off-the-shelf camera.
We propose an image-based, facial reenactment system that replaces the face of an actor in an existing target video with the face of a user from a source video, while preserving the original target performance. Our system is fully automatic and does not require a database of source expressions. Instead, it is able to produce convincing reenactment results from a short source video captured with an off-the-shelf camera, such as a webcam, where the user performs arbitrary facial gestures. Our reenactment pipeline is conceived as part image retrieval and part face transfer: The image retrieval is based on temporal clustering of target frames and a novel image matching metric that combines appearance and motion to select candidate frames from the source video, while the face transfer uses a 2D warping strategy that preserves the user's identity. Our system excels in simplicity as it does not rely on a 3D face model, it is robust under head motion and does not require the source and target performance to be similar. We show convincing reenactment results for videos that we recorded ourselves and for low-quality footage taken from the Internet.