Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
This work addresses the problem of robust face forgery detection for security and media integrity applications, representing an incremental improvement by using existing self-supervised techniques in a novel way.
The paper tackled the challenge of generalizing face forgery detection to unseen manipulation methods and maintaining robustness under corruptions like compression, by leveraging self-supervised learning from real talking face videos, achieving state-of-the-art performance in cross-manipulation generalization and robustness experiments.
One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we examine whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real videos to learn, in a self-supervised cross-modal manner, temporally dense video representations that capture factors such as facial movements, expression, and identity. Second, we use these learned representations as targets to be predicted by our forgery detector along with the usual binary forgery classification task; this encourages it to base its real/fake decision on said factors. We show that our method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments, and examine the factors that contribute to its performance. Our results suggest that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors.