SGSST: Scaling Gaussian Splatting StyleTransfer
This work addresses style transfer for 3D scenes, which is a domain-specific problem for computer graphics and neural rendering, and it is incremental as it builds on existing 3D Gaussian splatting techniques.
The paper tackles the problem of applying style transfer to full 3D environments by introducing SGSST, an optimization-based method that uses a multiscale loss to enable style transfer on ultra-high resolution 3D Gaussian splatting scenes, achieving superior visual quality as demonstrated through qualitative, quantitative, and perceptual comparisons.
Applying style transfer to a full 3D environment is a challenging task that has seen many developments since the advent of neural rendering. 3D Gaussian splatting (3DGS) has recently pushed further many limits of neural rendering in terms of training speed and reconstruction quality. This work introduces SGSST: Scaling Gaussian Splatting Style Transfer, an optimization-based method to apply style transfer to pretrained 3DGS scenes. We demonstrate that a new multiscale loss based on global neural statistics, that we name SOS for Simultaneously Optimized Scales, enables style transfer to ultra-high resolution 3D scenes. Not only SGSST pioneers 3D scene style transfer at such high image resolutions, it also produces superior visual quality as assessed by thorough qualitative, quantitative and perceptual comparisons.