ABC-GS: Alignment-Based Controllable Style Transfer for 3D Gaussian Splatting
This work provides a solution for controllable 3D style transfer, which is incremental as it builds on prior methods like NeRF and 3D Gaussian Splatting to improve stylization for applications in graphics and visualization.
The paper tackles the problem of 3D scene stylization by addressing limitations in existing Neural Radiance Fields (NeRF) methods, such as lack of global style consideration and fine-grained control, and introduces ABC-GS, a framework based on 3D Gaussian Splatting that achieves high-quality style transfer with controllability and better alignment to global style.
3D scene stylization approaches based on Neural Radiance Fields (NeRF) achieve promising results by optimizing with Nearest Neighbor Feature Matching (NNFM) loss. However, NNFM loss does not consider global style information. In addition, the implicit representation of NeRF limits their fine-grained control over the resulting scenes. In this paper, we introduce ABC-GS, a novel framework based on 3D Gaussian Splatting to achieve high-quality 3D style transfer. To this end, a controllable matching stage is designed to achieve precise alignment between scene content and style features through segmentation masks. Moreover, a style transfer loss function based on feature alignment is proposed to ensure that the outcomes of style transfer accurately reflect the global style of the reference image. Furthermore, the original geometric information of the scene is preserved with the depth loss and Gaussian regularization terms. Extensive experiments show that our ABC-GS provides controllability of style transfer and achieves stylization results that are more faithfully aligned with the global style of the chosen artistic reference. Our homepage is available at https://vpx-ecnu.github.io/ABC-GS-website.