Physics-Grounded Shadow Generation from Monocular 3D Geometry Priors and Approximate Light Direction
This work addresses the problem of generating photorealistic shadows in computer vision, offering a novel integration of physics and deep learning for improved accuracy in scenes with complex geometry or ambiguous lighting.
The paper tackles shadow generation by embedding explicit physical modeling of geometry and illumination into a deep learning framework, achieving visually realistic and physically coherent shadows that outperform existing methods, particularly in complex scenes.
Shadow generation aims to produce photorealistic shadows that are visually consistent with object geometry and scene illumination. In the physics of shadow formation, the occluder blocks some light rays casting from the light source that would otherwise arrive at the surface, creating a shadow that follows the silhouette of the occluder. However, such explicit physical modeling has rarely been used in deep-learning-based shadow generation. In this paper, we propose a novel framework that embeds explicit physical modeling - geometry and illumination - into deep-learning-based shadow generation. First, given a monocular RGB image, we obtain approximate 3D geometry in the form of dense point maps and predict a single dominant light direction. These signals allow us to recover fairly accurate shadow location and shape based on the physics of shadow formation. We then integrate this physics-based initial estimate into a diffusion framework that refines the shadow into a realistic, high-fidelity appearance while ensuring consistency with scene geometry and illumination. Trained on DESOBAV2, our model produces shadows that are both visually realistic and physically coherent, outperforming existing approaches, especially in scenes with complex geometry or ambiguous lighting.