Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models
This work addresses the threat of deepfake images to information authenticity, but it is incremental as it focuses on dataset creation and evaluation methods without introducing new detection algorithms.
The study tackled the problem of deepfake detection by creating the DeepFakeFace dataset using diffusion models and evaluating algorithms for robustness and generalizability across different fake generation methods and image imperfections, with results showing varied performance that highlights the need for improvement.
The rise of deepfake images, especially of well-known personalities, poses a serious threat to the dissemination of authentic information. To tackle this, we present a thorough investigation into how deepfakes are produced and how they can be identified. The cornerstone of our research is a rich collection of artificial celebrity faces, titled DeepFakeFace (DFF). We crafted the DFF dataset using advanced diffusion models and have shared it with the community through online platforms. This data serves as a robust foundation to train and test algorithms designed to spot deepfakes. We carried out a thorough review of the DFF dataset and suggest two evaluation methods to gauge the strength and adaptability of deepfake recognition tools. The first method tests whether an algorithm trained on one type of fake images can recognize those produced by other methods. The second evaluates the algorithm's performance with imperfect images, like those that are blurry, of low quality, or compressed. Given varied results across deepfake methods and image changes, our findings stress the need for better deepfake detectors. Our DFF dataset and tests aim to boost the development of more effective tools against deepfakes.