iVR-GS: Inverse Volume Rendering for Explorable Visualization via Editable 3D Gaussian Splatting
This work addresses the need for more flexible and efficient volume exploration tools for researchers and practitioners in fields like medical imaging or scientific visualization, though it is incremental as it builds on existing Gaussian splatting techniques.
The paper tackles the problem of limited user exploration in volume visualization due to fixed transfer function settings in existing novel view synthesis methods, introducing iVR-GS which enables interactive scene editing and achieves superior reconstruction quality compared to other solutions like Plenoxels and CCNeRF.
In volume visualization, users can interactively explore the three-dimensional data by specifying color and opacity mappings in the transfer function (TF) or adjusting lighting parameters, facilitating meaningful interpretation of the underlying structure. However, rendering large-scale volumes demands powerful GPUs and high-speed memory access for real-time performance. While existing novel view synthesis (NVS) methods offer faster rendering speeds with lower hardware requirements, the visible parts of a reconstructed scene are fixed and constrained by preset TF settings, significantly limiting user exploration. This paper introduces inverse volume rendering via Gaussian splatting (iVR-GS), an innovative NVS method that reduces the rendering cost while enabling scene editing for interactive volume exploration. Specifically, we compose multiple iVR-GS models associated with basic TFs covering disjoint visible parts to make the entire volumetric scene visible. Each basic model contains a collection of 3D editable Gaussians, where each Gaussian is a 3D spatial point that supports real-time scene rendering and editing. We demonstrate the superior reconstruction quality and composability of iVR-GS against other NVS solutions (Plenoxels, CCNeRF, and base 3DGS) on various volume datasets. The code is available at https://github.com/TouKaienn/iVR-GS.