A Large-scale Universal Evaluation Benchmark For Face Forgery Detection
This provides a standardized evaluation tool for researchers and practitioners in AI security to assess and improve detection methods against realistic fake facial content, though it is incremental as it builds on existing benchmark efforts.
The authors tackled the challenge of evaluating face forgery detection techniques by constructing DeepFaceGen, a large-scale benchmark with over 1.5 million real and fake samples from 34 generation methods, and used it to analyze 13 detection techniques, revealing significant findings and future directions.
With the rapid development of AI-generated content (AIGC) technology, the production of realistic fake facial images and videos that deceive human visual perception has become possible. Consequently, various face forgery detection techniques have been proposed to identify such fake facial content. However, evaluating the effectiveness and generalizability of these detection techniques remains a significant challenge. To address this, we have constructed a large-scale evaluation benchmark called DeepFaceGen, aimed at quantitatively assessing the effectiveness of face forgery detection and facilitating the iterative development of forgery detection technology. DeepFaceGen consists of 776,990 real face image/video samples and 773,812 face forgery image/video samples, generated using 34 mainstream face generation techniques. During the construction process, we carefully consider important factors such as content diversity, fairness across ethnicities, and availability of comprehensive labels, in order to ensure the versatility and convenience of DeepFaceGen. Subsequently, DeepFaceGen is employed in this study to evaluate and analyze the performance of 13 mainstream face forgery detection techniques from various perspectives. Through extensive experimental analysis, we derive significant findings and propose potential directions for future research. The code and dataset for DeepFaceGen are available at https://github.com/HengruiLou/DeepFaceGen.