Flexible Techniques for Differentiable Rendering with 3D Gaussians
This work addresses the need for efficient shape reconstruction in computer vision applications, but it is incremental as it builds on existing 3D Gaussian methods.
The paper tackles the problem of fast and reliable shape reconstruction for computer vision by extending differentiable rendering techniques based on 3D Gaussians, resulting in quick and robust reconstructions that are interoperable across methods and can be performed on GPU or CPU.
Fast, reliable shape reconstruction is an essential ingredient in many computer vision applications. Neural Radiance Fields demonstrated that photorealistic novel view synthesis is within reach, but was gated by performance requirements for fast reconstruction of real scenes and objects. Several recent approaches have built on alternative shape representations, in particular, 3D Gaussians. We develop extensions to these renderers, such as integrating differentiable optical flow, exporting watertight meshes and rendering per-ray normals. Additionally, we show how two of the recent methods are interoperable with each other. These reconstructions are quick, robust, and easily performed on GPU or CPU. For code and visual examples, see https://leonidk.github.io/fmb-plus