Decoupled Illumination Priors for Spatially Controllable Multi-View Indoor Scene Relighting
For computer graphics and vision researchers, this work addresses the challenge of precise 3D light placement in diffusion-based relighting without disrupting generative priors.
Lume-Palette enables spatially controllable multi-view indoor scene relighting by decoupling the process into illumination distillation and illumination casting, achieving photorealistic and consistent results on synthetic and real scenes.
Indoor scene relighting demands photorealism, precise spatial control, and strict multi-view consistency. While diffusion-based image editing models enable semantic lighting manipulation via text prompts, enforcing exact 3D light placement often disrupts their generative priors. We propose Lume-Palette, a progressive framework that leverages semantic lighting priors for spatially controllable multi-view indoor relighting. The approach decouples relighting into two stages: (1) illumination distillation, which extracts canonical illumination palettes from a pretrained diffusion model to preserve realistic material-light interactions, and (2) illumination casting, which explicitly maps target spatial lighting conditions defined from coarse 3D geometry. To efficiently handle dense multi-view and multi-modal inputs, we introduce an asymmetric multi-view conditioning strategy that selectively injects essential spatial context. Experiments on diverse synthetic scenes and real-world scenes demonstrate that Lume-Palette produces photorealistic, spatially controllable, and multi-view consistent relighting results. Project Page: https://cjeen.github.io/lumepalette