Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior
This work addresses the challenge of modeling natural illuminations for inverse rendering, which is incremental as it builds on prior methods like Vector Neurons and SIREN networks.
The authors tackled the ill-posed problem of inverse rendering by developing a rotation-equivariant neural illumination model to learn a prior for natural illuminations, resulting in a compact representation that expresses complex, high-frequency features and is demonstrated on tasks like environment map completion with a dataset of 1.6K HDR maps.
Inverse rendering is an ill-posed problem. Previous work has sought to resolve this by focussing on priors for object or scene shape or appearance. In this work, we instead focus on a prior for natural illuminations. Current methods rely on spherical harmonic lighting or other generic representations and, at best, a simplistic prior on the parameters. We propose a conditional neural field representation based on a variational auto-decoder with a SIREN network and, extending Vector Neurons, build equivariance directly into the network. Using this, we develop a rotation-equivariant, high dynamic range (HDR) neural illumination model that is compact and able to express complex, high-frequency features of natural environment maps. Training our model on a curated dataset of 1.6K HDR environment maps of natural scenes, we compare it against traditional representations, demonstrate its applicability for an inverse rendering task and show environment map completion from partial observations. A PyTorch implementation, our dataset and trained models can be found at jadgardner.github.io/RENI.