LightMover: Generative Light Movement with Color and Intensity Controls
This work addresses the problem of controllable light editing in single images for computer vision and graphics applications, offering a unified framework for spatial and appearance controls without re-rendering.
LightMover enables controllable light manipulation in single images by formulating light editing as a sequence-to-sequence prediction in visual token space, achieving precise control over light position, color, and intensity with physically plausible results, reducing control sequence length by 41% while maintaining high PSNR and semantic consistency.
We present LightMover, a framework for controllable light manipulation in single images that leverages video diffusion priors to produce physically plausible illumination changes without re-rendering the scene. We formulate light editing as a sequence-to-sequence prediction problem in visual token space: given an image and light-control tokens, the model adjusts light position, color, and intensity together with resulting reflections, shadows, and falloff from a single view. This unified treatment of spatial (movement) and appearance (color, intensity) controls improves both manipulation and illumination understanding. We further introduce an adaptive token-pruning mechanism that preserves spatially informative tokens while compactly encoding non-spatial attributes, reducing control sequence length by 41% while maintaining editing fidelity. To train our framework, we construct a scalable rendering pipeline that generates large numbers of image pairs across varied light positions, colors, and intensities while keeping the scene content consistent with the original image. LightMover enables precise, independent control over light position, color, and intensity, and achieves high PSNR and strong semantic consistency (DINO, CLIP) across different tasks.