No Shadow Left Behind: Removing Objects and their Shadows using Approximate Lighting and Geometry
This work is significant for mixed reality and image editing applications by enabling more believable object removal, addressing a limitation of existing inpainting techniques.
This paper addresses the problem of removing objects and their cast shadows from images, a task current inpainting methods struggle with. The authors developed a deep learning pipeline that leverages approximate lighting and geometry to remove a wide variety of shadows along with their casters, demonstrating results on both synthetic and real scenes.
Removing objects from images is a challenging problem that is important for many applications, including mixed reality. For believable results, the shadows that the object casts should also be removed. Current inpainting-based methods only remove the object itself, leaving shadows behind, or at best require specifying shadow regions to inpaint. We introduce a deep learning pipeline for removing a shadow along with its caster. We leverage rough scene models in order to remove a wide variety of shadows (hard or soft, dark or subtle, large or thin) from surfaces with a wide variety of textures. We train our pipeline on synthetically rendered data, and show qualitative and quantitative results on both synthetic and real scenes.