Differentiable Surface Rendering via Non-Differentiable Sampling
This method addresses the challenge of making 3D surface rendering differentiable for computer graphics and vision researchers, offering a fast and simple solution that supports explicit and implicit representations, though it appears incremental by building on existing rendering techniques.
The paper tackles the problem of differentiable rendering of 3D surfaces by introducing a method that uses non-differentiable sampling followed by differentiable point splatting, enabling efficient rendering for various surface representations and applications like inverse rendering and neural network training.
We present a method for differentiable rendering of 3D surfaces that supports both explicit and implicit representations, provides derivatives at occlusion boundaries, and is fast and simple to implement. The method first samples the surface using non-differentiable rasterization, then applies differentiable, depth-aware point splatting to produce the final image. Our approach requires no differentiable meshing or rasterization steps, making it efficient for large 3D models and applicable to isosurfaces extracted from implicit surface definitions. We demonstrate the effectiveness of our method for implicit-, mesh-, and parametric-surface-based inverse rendering and neural-network training applications. In particular, we show for the first time efficient, differentiable rendering of an isosurface extracted from a neural radiance field (NeRF), and demonstrate surface-based, rather than volume-based, rendering of a NeRF.