Learning from Synthetic Shadows for Shadow Detection and Removal
This work provides a solution to the data scarcity problem for researchers and practitioners working on shadow detection and removal, which is a common issue in computer vision and graphics.
This paper addresses the challenge of limited real paired shadow/shadow-free datasets for training shadow detection and removal models. The authors introduce SynShadow, a large-scale synthetic dataset generated using an extended physically-grounded shadow illumination model. Models trained on SynShadow demonstrate improved performance on challenging benchmarks and can be used for fine-tuning to enhance existing shadow detection and removal models.
Shadow removal is an essential task in computer vision and computer graphics. Recent shadow removal approaches all train convolutional neural networks (CNN) on real paired shadow/shadow-free or shadow/shadow-free/mask image datasets. However, obtaining a large-scale, diverse, and accurate dataset has been a big challenge, and it limits the performance of the learned models on shadow images with unseen shapes/intensities. To overcome this challenge, we present SynShadow, a novel large-scale synthetic shadow/shadow-free/matte image triplets dataset and a pipeline to synthesize it. We extend a physically-grounded shadow illumination model and synthesize a shadow image given an arbitrary combination of a shadow-free image, a matte image, and shadow attenuation parameters. Owing to the diversity, quantity, and quality of SynShadow, we demonstrate that shadow removal models trained on SynShadow perform well in removing shadows with diverse shapes and intensities on some challenging benchmarks. Furthermore, we show that merely fine-tuning from a SynShadow-pre-trained model improves existing shadow detection and removal models. Codes are publicly available at https://github.com/naoto0804/SynShadow.