SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data
This addresses the challenge of reconstructing reflective objects from limited views for applications in computer vision and graphics, representing an incremental improvement over existing neural rendering methods.
The paper tackles the problem of 3D reconstruction of partly-symmetric objects from incomplete data, such as in-the-wild images, by using a soft symmetry constraint on geometry and materials, and shows results of high-fidelity reconstruction of unobserved regions and high-quality novel view rendering on the CO3D car dataset.
We present a method for the accurate 3D reconstruction of partly-symmetric objects. We build on the strengths of recent advances in neural reconstruction and rendering such as Neural Radiance Fields (NeRF). A major shortcoming of such approaches is that they fail to reconstruct any part of the object which is not clearly visible in the training image, which is often the case for in-the-wild images and videos. When evidence is lacking, structural priors such as symmetry can be used to complete the missing information. However, exploiting such priors in neural rendering is highly non-trivial: while geometry and non-reflective materials may be symmetric, shadows and reflections from the ambient scene are not symmetric in general. To address this, we apply a soft symmetry constraint to the 3D geometry and material properties, having factored appearance into lighting, albedo colour and reflectivity. We evaluate our method on the recently introduced CO3D dataset, focusing on the car category due to the challenge of reconstructing highly-reflective materials. We show that it can reconstruct unobserved regions with high fidelity and render high-quality novel view images.