Detect Any Deepfakes: Segment Anything Meets Face Forgery Detection and Localization
This work addresses the challenge of accurately detecting and localizing manipulated areas in deepfakes, which is crucial for security and media integrity, though it is incremental as it builds on existing foundation models.
The paper tackles the problem of precise deepfake detection and localization with limited pixel-wise supervision by integrating the Segment Anything Model (SAM) into a framework called Detect Any Deepfakes (DADF), achieving superior performance on three benchmark datasets.
The rapid advancements in computer vision have stimulated remarkable progress in face forgery techniques, capturing the dedicated attention of researchers committed to detecting forgeries and precisely localizing manipulated areas. Nonetheless, with limited fine-grained pixel-wise supervision labels, deepfake detection models perform unsatisfactorily on precise forgery detection and localization. To address this challenge, we introduce the well-trained vision segmentation foundation model, i.e., Segment Anything Model (SAM) in face forgery detection and localization. Based on SAM, we propose the Detect Any Deepfakes (DADF) framework with the Multiscale Adapter, which can capture short- and long-range forgery contexts for efficient fine-tuning. Moreover, to better identify forged traces and augment the model's sensitivity towards forgery regions, Reconstruction Guided Attention (RGA) module is proposed. The proposed framework seamlessly integrates end-to-end forgery localization and detection optimization. Extensive experiments on three benchmark datasets demonstrate the superiority of our approach for both forgery detection and localization. The codes will be released soon at https://github.com/laiyingxin2/DADF.