Learning Illumination Control in Diffusion Models
This work provides a fully reproducible, open-source solution for illumination control in image generation, addressing the lack of open alternatives to closed-source models.
The authors present an open-source pipeline for illumination control in diffusion models, using a data engine to generate training triplets and finetuning models to achieve significant improvements over SD 1.5, SDXL, and FLUX.1-dev in perceptual similarity, structural similarity, and identity preservation.
Controlling illumination in images is essential for photography and visual content creation. While closed-source models have demonstrated impressive illumination control, open-source alternatives either require heavy control inputs like depth maps or do not release their data and code. We present a fully open-source and reproducible pipeline for learning illumination control in diffusion models. Our approach builds a data engine that transforms well-lit images into supervised training triplets consisting of a poorly-illuminated input image, a natural language lighting instruction, and a well-illuminated output image. We finetune a diffusion model on this data and demonstrate significant improvements over baseline SD 1.5, SDXL, and FLUX.1-dev models in perceptual similarity, structural similarity, and identity preservation. Our work provides a reproducible solution built entirely with open-source tools and publicly available data. We release all our code, data, and model weights publicly.