Beyond Detection: Multi-Scale Hidden-Code for Natural Image Deepfake Recovery and Factual Retrieval
This work is significant for anyone needing to recover original content from deepfakes, moving beyond just identifying them, which is an incremental but important step for digital forensics and content verification.
This paper addresses the underexplored problem of recovering tampered content from deepfakes for factual retrieval, rather than just detection. The authors propose a unified hidden-code recovery framework that encodes semantic and perceptual information, refined through multi-scale vector quantization, and uses conditional Transformer modules for contextual reasoning. They introduce ImageNet-S, a benchmark for paired image-label factual retrieval, and demonstrate promising retrieval and reconstruction performance on it.
Recent advances in image authenticity have primarily focused on deepfake detection and localization, leaving recovery of tampered contents for factual retrieval relatively underexplored. We propose a unified hidden-code recovery framework that enables both retrieval and restoration from post-hoc and in-generation watermarking paradigms. Our method encodes semantic and perceptual information into a compact hidden-code representation, refined through multi-scale vector quantization, and enhances contextual reasoning via conditional Transformer modules. To enable systematic evaluation for natural images, we construct ImageNet-S, a benchmark that provides paired image-label factual retrieval tasks. Extensive experiments on ImageNet-S demonstrate that our method exhibits promising retrieval and reconstruction performance while remaining fully compatible with diverse watermarking pipelines. This framework establishes a foundation for general-purpose image recovery beyond detection and localization.