Detecting Deepfakes with Self-Blended Images
This addresses the challenge of deepfake detection for security and media integrity, offering an incremental improvement in generalization over existing methods.
The paper tackles the problem of detecting deepfakes by introducing self-blended images (SBIs) as synthetic training data, which improves model generalization to unknown manipulations and scenes, achieving performance gains of 4.90% and 11.78% on DFDC and DFDCP datasets in cross-dataset evaluations.
In this paper, we present novel synthetic training data called self-blended images (SBIs) to detect deepfakes. SBIs are generated by blending pseudo source and target images from single pristine images, reproducing common forgery artifacts (e.g., blending boundaries and statistical inconsistencies between source and target images). The key idea behind SBIs is that more general and hardly recognizable fake samples encourage classifiers to learn generic and robust representations without overfitting to manipulation-specific artifacts. We compare our approach with state-of-the-art methods on FF++, CDF, DFD, DFDC, DFDCP, and FFIW datasets by following the standard cross-dataset and cross-manipulation protocols. Extensive experiments show that our method improves the model generalization to unknown manipulations and scenes. In particular, on DFDC and DFDCP where existing methods suffer from the domain gap between the training and test sets, our approach outperforms the baseline by 4.90% and 11.78% points in the cross-dataset evaluation, respectively.