Lighting (In)consistency of Paint by Text
This addresses the challenge of detecting paint-by-text synthetic media for the photo-forensic community, but it is incremental as it provides only an initial exploration.
The paper investigated whether physics-based forensic analysis can detect synthetic images generated by DALL-E-2, focusing on lighting inconsistencies, and found initial evidence suggesting potential for detection.
Whereas generative adversarial networks are capable of synthesizing highly realistic images of faces, cats, landscapes, or almost any other single category, paint-by-text synthesis engines can -- from a single text prompt -- synthesize realistic images of seemingly endless categories with arbitrary configurations and combinations. This powerful technology poses new challenges to the photo-forensic community. Motivated by the fact that paint by text is not based on explicit geometric or physical models, and the human visual system's general insensitivity to lighting inconsistencies, we provide an initial exploration of the lighting consistency of DALL-E-2 synthesized images to determine if physics-based forensic analyses will prove fruitful in detecting this new breed of synthetic media.