ROGR: Relightable 3D Objects using Generative Relighting
This addresses the problem of efficient 3D object relighting for computer graphics and vision applications, representing a novel method for a known bottleneck.
The paper tackles the problem of reconstructing relightable 3D objects from multiple views by introducing ROGR, which uses a generative relighting model to train a lighting-conditioned Neural Radiance Field (NeRF) that outputs appearance under arbitrary lighting. It improves upon state-of-the-art on most metrics on TensoIR and Stanford-ORB datasets.
We introduce ROGR, a novel approach that reconstructs a relightable 3D model of an object captured from multiple views, driven by a generative relighting model that simulates the effects of placing the object under novel environment illuminations. Our method samples the appearance of the object under multiple lighting environments, creating a dataset that is used to train a lighting-conditioned Neural Radiance Field (NeRF) that outputs the object's appearance under any input environmental lighting. The lighting-conditioned NeRF uses a novel dual-branch architecture to encode the general lighting effects and specularities separately. The optimized lighting-conditioned NeRF enables efficient feed-forward relighting under arbitrary environment maps without requiring per-illumination optimization or light transport simulation. We evaluate our approach on the established TensoIR and Stanford-ORB datasets, where it improves upon the state-of-the-art on most metrics, and showcase our approach on real-world object captures.