Fast Face-swap Using Convolutional Neural Networks
This addresses the problem of automated face swapping for applications like entertainment or security, but it is incremental as it builds on existing style transfer methods.
The paper tackles face swapping in images by transforming an input identity into a target identity while preserving pose, expression, and lighting, using convolutional neural networks trained on unstructured photo collections to achieve photorealistic results in real-time without user input.
We consider the problem of face swapping in images, where an input identity is transformed into a target identity while preserving pose, facial expression, and lighting. To perform this mapping, we use convolutional neural networks trained to capture the appearance of the target identity from an unstructured collection of his/her photographs.This approach is enabled by framing the face swapping problem in terms of style transfer, where the goal is to render an image in the style of another one. Building on recent advances in this area, we devise a new loss function that enables the network to produce highly photorealistic results. By combining neural networks with simple pre- and post-processing steps, we aim at making face swap work in real-time with no input from the user.