GGViT:Multistream Vision Transformer Network in Face2Face Facial Reenactment Detection
This addresses the urgent need to detect face2face facial reenactment forgeries on social networks, especially under varying video compression, but it is incremental as it builds on existing transformer and multi-stream architectures.
The paper tackles the problem of detecting manipulated facial videos on social media, where compression destroys pixel details, by proposing GGViT, a multi-stream vision transformer network that achieves state-of-the-art accuracy on the FF++ dataset, with improvements of 24.34%, 15.08%, and 10.14% on specific compression scenarios.
Detecting manipulated facial images and videos on social networks has been an urgent problem to be solved. The compression of videos on social media has destroyed some pixel details that could be used to detect forgeries. Hence, it is crucial to detect manipulated faces in videos of different quality. We propose a new multi-stream network architecture named GGViT, which utilizes global information to improve the generalization of the model. The embedding of the whole face extracted by ViT will guide each stream network. Through a large number of experiments, we have proved that our proposed model achieves state-of-the-art classification accuracy on FF++ dataset, and has been greatly improved on scenarios of different compression rates. The accuracy of Raw/C23, Raw/C40 and C23/C40 was increased by 24.34%, 15.08% and 10.14% respectively.