Hardware-Rasterized Ray-Based Gaussian Splatting
This enables real-time, high-quality novel view synthesis for Virtual and Mixed Reality applications, representing a significant performance improvement over existing methods.
The paper tackles the problem of slow rendering speeds for ray-based 3D Gaussian Splatting (RayGS) models by introducing a hardware-rasterized approach, achieving high frame rates suitable for VR/MR applications while maintaining state-of-the-art quality. It also addresses alias-free rendering by solving MIP-related issues during training and testing.
We present a novel, hardware rasterized rendering approach for ray-based 3D Gaussian Splatting (RayGS), obtaining both fast and high-quality results for novel view synthesis. Our work contains a mathematically rigorous and geometrically intuitive derivation about how to efficiently estimate all relevant quantities for rendering RayGS models, structured with respect to standard hardware rasterization shaders. Our solution is the first enabling rendering RayGS models at sufficiently high frame rates to support quality-sensitive applications like Virtual and Mixed Reality. Our second contribution enables alias-free rendering for RayGS, by addressing MIP-related issues arising when rendering diverging scales during training and testing. We demonstrate significant performance gains, across different benchmark scenes, while retaining state-of-the-art appearance quality of RayGS.