I$^2$-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs
This work addresses the challenge of creating editable and photorealistic indoor scene models for applications in computer vision and graphics, representing an incremental improvement through novel techniques like bubble loss and adaptive sampling.
The paper tackles the problem of reconstructing and editing indoor scenes from multi-view images by jointly recovering shapes, radiance, and materials using neural signed distance fields with differentiable raytracing, achieving superior quality in reconstruction, novel view synthesis, and editing compared to state-of-the-art methods.
In this work, we present I$^2$-SDF, a new method for intrinsic indoor scene reconstruction and editing using differentiable Monte Carlo raytracing on neural signed distance fields (SDFs). Our holistic neural SDF-based framework jointly recovers the underlying shapes, incident radiance and materials from multi-view images. We introduce a novel bubble loss for fine-grained small objects and error-guided adaptive sampling scheme to largely improve the reconstruction quality on large-scale indoor scenes. Further, we propose to decompose the neural radiance field into spatially-varying material of the scene as a neural field through surface-based, differentiable Monte Carlo raytracing and emitter semantic segmentations, which enables physically based and photorealistic scene relighting and editing applications. Through a number of qualitative and quantitative experiments, we demonstrate the superior quality of our method on indoor scene reconstruction, novel view synthesis, and scene editing compared to state-of-the-art baselines.