GS^3: Efficient Relighting with Triple Gaussian Splatting
This work addresses the challenge of efficient relighting for applications like virtual reality and graphics, though it is incremental as it builds on existing Gaussian splatting methods.
The paper tackles the problem of real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit images by introducing a spatial and angular Gaussian representation with a triple splatting process, achieving a training time of 40-70 minutes and rendering at 90 fps on a single GPU.
We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex appearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance. Our code and data are publicly available at https://GSrelight.github.io/.