Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection
This addresses the generalization problem in deepfake detection for security and media verification applications, representing an incremental advance with a novel training approach.
The paper tackles the challenge of generalizing deepfake detectors to unseen datasets and methods by proposing Curricular Dynamic Forgery Augmentation (CDFA), which uses a curriculum-based and dynamic augmentation strategy during training, resulting in significant improvements in cross-dataset and cross-manipulation performance for various detectors.
Previous studies in deepfake detection have shown promising results when testing face forgeries from the same dataset as the training. However, the problem remains challenging when one tries to generalize the detector to forgeries from unseen datasets and created by unseen methods. In this work, we present a novel general deepfake detection method, called \textbf{C}urricular \textbf{D}ynamic \textbf{F}orgery \textbf{A}ugmentation (CDFA), which jointly trains a deepfake detector with a forgery augmentation policy network. Unlike the previous works, we propose to progressively apply forgery augmentations following a monotonic curriculum during the training. We further propose a dynamic forgery searching strategy to select one suitable forgery augmentation operation for each image varying between training stages, producing a forgery augmentation policy optimized for better generalization. In addition, we propose a novel forgery augmentation named self-shifted blending image to simply imitate the temporal inconsistency of deepfake generation. Comprehensive experiments show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors in a plug-and-play way, and make them attain superior performances over the existing methods in several benchmark datasets.