Towards More Accurate Fake Detection on Images Generated from Advanced Generative and Neural Rendering Models
This addresses the need for robust fake detection in images from neural rendering techniques, which is an incremental improvement over existing methods for GANs and diffusion models.
The paper tackles the problem of detecting fake images generated by advanced neural rendering models like Neural Radiance Fields and 3D Gaussian splatting, proposing an unsupervised training technique that combines Fourier spectrum and spatial domain features to create a robust multimodal detector, and it also develops a comprehensive database of such images for evaluation.
The remarkable progress in neural-network-driven visual data generation, especially with neural rendering techniques like Neural Radiance Fields and 3D Gaussian splatting, offers a powerful alternative to GANs and diffusion models. These methods can produce high-fidelity images and lifelike avatars, highlighting the need for robust detection methods. In response, an unsupervised training technique is proposed that enables the model to extract comprehensive features from the Fourier spectrum magnitude, thereby overcoming the challenges of reconstructing the spectrum due to its centrosymmetric properties. By leveraging the spectral domain and dynamically combining it with spatial domain information, we create a robust multimodal detector that demonstrates superior generalization capabilities in identifying challenging synthetic images generated by the latest image synthesis techniques. To address the absence of a 3D neural rendering-based fake image database, we develop a comprehensive database that includes images generated by diverse neural rendering techniques, providing a robust foundation for evaluating and advancing detection methods.