Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory
This addresses the challenge of precise lighting manipulation in generative AI for applications in computer graphics and visual effects, representing an incremental improvement by integrating Retinex theory into existing diffusion frameworks.
The paper tackles the problem of controlling illumination conditions in diffusion models for image generation, achieving realistic lighting effects like cast shadows and inter-reflections without requiring intrinsic decomposition, latent space exploration, or additional training.
This paper introduces a novel approach to illumination manipulation in diffusion models, addressing the gap in conditional image generation with a focus on lighting conditions. We conceptualize the diffusion model as a black-box image render and strategically decompose its energy function in alignment with the image formation model. Our method effectively separates and controls illumination-related properties during the generative process. It generates images with realistic illumination effects, including cast shadow, soft shadow, and inter-reflections. Remarkably, it achieves this without the necessity for learning intrinsic decomposition, finding directions in latent space, or undergoing additional training with new datasets.