When Gaussian Meets Surfel: Ultra-fast High-fidelity Radiance Field Rendering
This addresses the speed-quality trade-off in neural radiance field rendering for applications like real-time graphics and VR, representing an incremental improvement over existing methods.
The paper tackles the problem of achieving both high-fidelity and ultra-fast rendering in radiance field representations by introducing Gaussian-enhanced Surfels (GESs), which combine 2D surfels for coarse geometry with 3D Gaussians for fine details, resulting in sorting-free rendering that avoids popping artifacts and achieves state-of-the-art performance.
We introduce Gaussian-enhanced Surfels (GESs), a bi-scale representation for radiance field rendering, wherein a set of 2D opaque surfels with view-dependent colors represent the coarse-scale geometry and appearance of scenes, and a few 3D Gaussians surrounding the surfels supplement fine-scale appearance details. The rendering with GESs consists of two passes -- surfels are first rasterized through a standard graphics pipeline to produce depth and color maps, and then Gaussians are splatted with depth testing and color accumulation on each pixel order independently. The optimization of GESs from multi-view images is performed through an elaborate coarse-to-fine procedure, faithfully capturing rich scene appearance. The entirely sorting-free rendering of GESs not only achieves very fast rates, but also produces view-consistent images, successfully avoiding popping artifacts under view changes. The basic GES representation can be easily extended to achieve anti-aliasing in rendering (Mip-GES), boosted rendering speeds (Speedy-GES) and compact storage (Compact-GES), and reconstruct better scene geometries by replacing 3D Gaussians with 2D Gaussians (2D-GES). Experimental results show that GESs advance the state-of-the-arts as a compelling representation for ultra-fast high-fidelity radiance field rendering.