Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
This addresses the need for trustworthy and interpretable deepfake detection systems for users and supervisors, though it is incremental as it builds on existing temporal artifact analysis.
The paper tackles the problem of detecting deepfakes by focusing on temporal inconsistencies in videos, proposing a method that achieves competitive performance on unseen datasets like Google's DeepFakeDetection, DeeperForensics, and Celeb-DF while providing interpretable visual explanations.
In this paper we propose a novel human-centered approach for detecting forgery in face images, using dynamic prototypes as a form of visual explanations. Currently, most state-of-the-art deepfake detections are based on black-box models that process videos frame-by-frame for inference, and few closely examine their temporal inconsistencies. However, the existence of such temporal artifacts within deepfake videos is key in detecting and explaining deepfakes to a supervising human. To this end, we propose Dynamic Prototype Network (DPNet) -- an interpretable and effective solution that utilizes dynamic representations (i.e., prototypes) to explain deepfake temporal artifacts. Extensive experimental results show that DPNet achieves competitive predictive performance, even on unseen testing datasets such as Google's DeepFakeDetection, DeeperForensics, and Celeb-DF, while providing easy referential explanations of deepfake dynamics. On top of DPNet's prototypical framework, we further formulate temporal logic specifications based on these dynamics to check our model's compliance to desired temporal behaviors, hence providing trustworthiness for such critical detection systems.