Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy
This addresses the need for high-quality, interactive anatomical visualization in medical imaging, though it is incremental as it builds on existing methods like path tracing and Gaussian Splatting.
The paper tackles the problem of computationally expensive and non-interactive volumetric medical visualization by proposing a hybrid rendering approach that combines foveated path tracing with peripheral Gaussian Splatting, achieving peripheral model regeneration in under a second and eliminating extensive preprocessing.
Volumetric medical imaging offers great potential for understanding complex pathologies. Yet, traditional 2D slices provide little support for interpreting spatial relationships, forcing users to mentally reconstruct anatomy into three dimensions. Direct volumetric path tracing and VR rendering can improve perception but are computationally expensive, while precomputed representations, like Gaussian Splatting, require planning ahead. Both approaches limit interactive use. We propose a hybrid rendering approach for high-quality, interactive, and immersive anatomical visualization. Our method combines streamed foveated path tracing with a lightweight Gaussian Splatting approximation of the periphery. The peripheral model generation is optimized with volume data and continuously refined using foveal renderings, enabling interactive updates. Depth-guided reprojection further improves robustness to latency and allows users to balance fidelity with refresh rate. We compare our method against direct path tracing and Gaussian Splatting. Our results highlight how their combination can preserve strengths in visual quality while re-generating the peripheral model in under a second, eliminating extensive preprocessing and approximations. This opens new options for interactive medical visualization.