Autonomous and Self-Adapting System for Synthetic Media Detection and Attribution
This addresses the critical need for forensic systems that can keep pace with rapidly evolving generative AI to combat disinformation and fraud.
The paper tackles the problem of synthetic image detection systems becoming obsolete as new generative models emerge by introducing an autonomous self-adaptive system that detects synthetic images, attributes them to known sources, and autonomously identifies and incorporates novel generators without human intervention. The method significantly outperforms existing approaches in experiments.
Rapid advances in generative AI have enabled the creation of highly realistic synthetic images, which, while beneficial in many domains, also pose serious risks in terms of disinformation, fraud, and other malicious applications. Current synthetic image identification systems are typically static, relying on feature representations learned from known generators; as new generative models emerge, these systems suffer from severe performance degradation. In this paper, we introduce the concept of an autonomous self-adaptive synthetic media identification system -- one that not only detects synthetic images and attributes them to known sources but also autonomously identifies and incorporates novel generators without human intervention. Our approach leverages an open-set identification strategy with an evolvable embedding space that distinguishes between known and unknown sources. By employing an unsupervised clustering method to aggregate unknown samples into high-confidence clusters and continuously refining its decision boundaries, our system maintains robust detection and attribution performance even as the generative landscape evolves. Extensive experiments demonstrate that our method significantly outperforms existing approaches, marking a crucial step toward universal, adaptable forensic systems in the era of rapidly advancing generative models.