ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
This addresses the challenge of transferability in image forgery detection for security and media verification, though it is incremental as it builds on existing domain adaptation and anomaly detection methods.
The paper tackles the problem of detecting manipulated images across unseen forgery methods by introducing Forensic-Transfer, a weakly-supervised domain adaptation approach that learns a forensic embedding for anomaly detection, achieving up to 85% accuracy on unseen examples and around 95% with only a few seen examples.
Distinguishing manipulated from real images is becoming increasingly difficult as new sophisticated image forgery approaches come out by the day. Naive classification approaches based on Convolutional Neural Networks (CNNs) show excellent performance in detecting image manipulations when they are trained on a specific forgery method. However, on examples from unseen manipulation approaches, their performance drops significantly. To address this limitation in transferability, we introduce Forensic-Transfer (FT). We devise a learning-based forensic detector which adapts well to new domains, i.e., novel manipulation methods and can handle scenarios where only a handful of fake examples are available during training. To this end, we learn a forensic embedding based on a novel autoencoder-based architecture that can be used to distinguish between real and fake imagery. The learned embedding acts as a form of anomaly detector; namely, an image manipulated from an unseen method will be detected as fake provided it maps sufficiently far away from the cluster of real images. Comparing to prior works, FT shows significant improvements in transferability, which we demonstrate in a series of experiments on cutting-edge benchmarks. For instance, on unseen examples, we achieve up to 85% in terms of accuracy, and with only a handful of seen examples, our performance already reaches around 95%.