DeRenderNet: Intrinsic Image Decomposition of Urban Scenes with Shape-(In)dependent Shading Rendering
This addresses the problem of accurate scene understanding for urban environments, offering an incremental improvement over existing intrinsic image decomposition methods.
The paper tackles intrinsic image decomposition for urban scenes by proposing DeRenderNet, a self-supervised deep neural network that decomposes albedo and latent lighting from a single image, using video game data for supervision; it achieves shadow-free albedo maps with clean details and accurate shadow prediction, improving re-rendering and high-level vision tasks.
We propose DeRenderNet, a deep neural network to decompose the albedo and latent lighting, and render shape-(in)dependent shadings, given a single image of an outdoor urban scene, trained in a self-supervised manner. To achieve this goal, we propose to use the albedo maps extracted from scenes in videogames as direct supervision and pre-compute the normal and shadow prior maps based on the depth maps provided as indirect supervision. Compared with state-of-the-art intrinsic image decomposition methods, DeRenderNet produces shadow-free albedo maps with clean details and an accurate prediction of shadows in the shape-independent shading, which is shown to be effective in re-rendering and improving the accuracy of high-level vision tasks for urban scenes.