Enhancing General Face Forgery Detection via Vision Transformer with Low-Rank Adaptation
This addresses security concerns from fake faces in applications like fraud, but it is incremental as it builds on existing vision transformer and adaptation techniques.
The paper tackled the problem of poor generalization in face forgery detection by proposing a vision transformer model with low-rank adaptation and single center loss, achieving state-of-the-art performance in cross-manipulation and cross-dataset evaluations.
Nowadays, forgery faces pose pressing security concerns over fake news, fraud, impersonation, etc. Despite the demonstrated success in intra-domain face forgery detection, existing detection methods lack generalization capability and tend to suffer from dramatic performance drops when deployed to unforeseen domains. To mitigate this issue, this paper designs a more general fake face detection model based on the vision transformer(ViT) architecture. In the training phase, the pretrained ViT weights are freezed, and only the Low-Rank Adaptation(LoRA) modules are updated. Additionally, the Single Center Loss(SCL) is applied to supervise the training process, further improving the generalization capability of the model. The proposed method achieves state-of-the-arts detection performances in both cross-manipulation and cross-dataset evaluations.