IBGS: Image-Based Gaussian Splatting
This addresses the problem of improving novel view synthesis quality for 3D scene rendering, though it is incremental as it builds on existing Gaussian Splatting methods.
The paper tackled the problem of 3D Gaussian Splatting's limitations in capturing spatially varying color and view-dependent effects like specular highlights by proposing Image-Based Gaussian Splatting, which leverages high-resolution source images to model pixel colors as a combination of base color and learned residuals, resulting in significantly outperforming prior methods in rendering quality on standard benchmarks without increasing storage.
3D Gaussian Splatting (3DGS) has recently emerged as a fast, high-quality method for novel view synthesis (NVS). However, its use of low-degree spherical harmonics limits its ability to capture spatially varying color and view-dependent effects such as specular highlights. Existing works augment Gaussians with either a global texture map, which struggles with complex scenes, or per-Gaussian texture maps, which introduces high storage overhead. We propose Image-Based Gaussian Splatting, an efficient alternative that leverages high-resolution source images for fine details and view-specific color modeling. Specifically, we model each pixel color as a combination of a base color from standard 3DGS rendering and a learned residual inferred from neighboring training images. This promotes accurate surface alignment and enables rendering images of high-frequency details and accurate view-dependent effects. Experiments on standard NVS benchmarks show that our method significantly outperforms prior Gaussian Splatting approaches in rendering quality, without increasing the storage footprint.