Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution
This work addresses the challenge of deploying face swap on resource-constrained platforms, offering a practical solution for mobile applications.
The authors tackled the problem of enabling face swap on mobile devices by proposing MobileFSGAN, a lightweight GAN that reduces parameters to 10.2MB and runs at real-time speed while achieving competitive performance compared to state-of-the-art methods.
Existing face swap methods rely heavily on large-scale networks for adequate capacity to generate visually plausible results, which inhibits its applications on resource-constraint platforms. In this work, we propose MobileFSGAN, a novel lightweight GAN for face swap that can run on mobile devices with much fewer parameters while achieving competitive performance. A lightweight encoder-decoder structure is designed especially for image synthesis tasks, which is only 10.2MB and can run on mobile devices at a real-time speed. To tackle the unstability of training such a small network, we construct the FSTriplets dataset utilizing facial attribute editing techniques. FSTriplets provides source-target-result training triplets, yielding pixel-level labels thus for the first time making the training process supervised. We also designed multi-scale gradient losses for efficient back-propagation, resulting in faster and better convergence. Experimental results show that our model reaches comparable performance towards state-of-the-art methods, while significantly reducing the number of network parameters. Codes and the dataset have been released.