Render-FM: A Foundation Model for Real-time Photorealistic Volumetric Rendering
This enables real-time interactive 3D visualizations for clinical applications like surgical planning and diagnostics, representing a strong domain-specific advancement.
The paper tackles the problem of slow, per-scene optimization in volumetric rendering of CT scans for medical imaging by proposing Render-FM, a foundation model that directly regresses 6D Gaussian Splatting parameters, reducing preparation time from nearly an hour to seconds while achieving comparable or superior visual fidelity.
Volumetric rendering of Computed Tomography (CT) scans is crucial for visualizing complex 3D anatomical structures in medical imaging. Current high-fidelity approaches, especially neural rendering techniques, require time-consuming per-scene optimization, limiting clinical applicability due to computational demands and poor generalizability. We propose Render-FM, a novel foundation model for direct, real-time volumetric rendering of CT scans. Render-FM employs an encoder-decoder architecture that directly regresses 6D Gaussian Splatting (6DGS) parameters from CT volumes, eliminating per-scan optimization through large-scale pre-training on diverse medical data. By integrating robust feature extraction with the expressive power of 6DGS, our approach efficiently generates high-quality, real-time interactive 3D visualizations across diverse clinical CT data. Experiments demonstrate that Render-FM achieves visual fidelity comparable or superior to specialized per-scan methods while drastically reducing preparation time from nearly an hour to seconds for a single inference step. This advancement enables seamless integration into real-time surgical planning and diagnostic workflows. The project page is: https://gaozhongpai.github.io/renderfm/.