De-rendering the World's Revolutionary Artefacts
This work addresses the challenge of decomposing 3D shape, appearance, and lighting for rotationally symmetric artefacts with complex surface properties, which is incremental as it builds on prior unsupervised methods by handling more realistic materials.
The paper tackles the problem of unsupervised image de-rendering for real-world artefacts with complex materials like specular reflections, proposing RADAR to recover environment illumination and surface materials from single-image collections without explicit 3D supervision, and demonstrates compelling results on a real vase dataset enabling applications such as free-viewpoint rendering and relighting.
Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision. However, many of these assume simplistic material and lighting models. We propose a method, termed RADAR, that can recover environment illumination and surface materials from real single-image collections, relying neither on explicit 3D supervision, nor on multi-view or multi-light images. Specifically, we focus on rotationally symmetric artefacts that exhibit challenging surface properties including specular reflections, such as vases. We introduce a novel self-supervised albedo discriminator, which allows the model to recover plausible albedo without requiring any ground-truth during training. In conjunction with a shape reconstruction module exploiting rotational symmetry, we present an end-to-end learning framework that is able to de-render the world's revolutionary artefacts. We conduct experiments on a real vase dataset and demonstrate compelling decomposition results, allowing for applications including free-viewpoint rendering and relighting.