Towards Universal Fake Image Detectors that Generalize Across Generative Models
This addresses the need for universal fake image detectors that work across proliferating generative models, offering a novel approach with strong generalization gains.
The paper tackles the problem of fake image detectors failing to generalize across new generative models by showing that existing trained classifiers become asymmetrically tuned, and proposes using a feature space from a pretrained vision-language model with nearest neighbor classification, achieving improvements of +15.07 mAP and +25.90% accuracy on unseen models.
With generative models proliferating at a rapid rate, there is a growing need for general purpose fake image detectors. In this work, we first show that the existing paradigm, which consists of training a deep network for real-vs-fake classification, fails to detect fake images from newer breeds of generative models when trained to detect GAN fake images. Upon analysis, we find that the resulting classifier is asymmetrically tuned to detect patterns that make an image fake. The real class becomes a sink class holding anything that is not fake, including generated images from models not accessible during training. Building upon this discovery, we propose to perform real-vs-fake classification without learning; i.e., using a feature space not explicitly trained to distinguish real from fake images. We use nearest neighbor and linear probing as instantiations of this idea. When given access to the feature space of a large pretrained vision-language model, the very simple baseline of nearest neighbor classification has surprisingly good generalization ability in detecting fake images from a wide variety of generative models; e.g., it improves upon the SoTA by +15.07 mAP and +25.90% acc when tested on unseen diffusion and autoregressive models.