BridgeSplat: Bidirectionally Coupled CT and Non-Rigid Gaussian Splatting for Deformable Intraoperative Surgical Navigation
This addresses the challenge of aligning surgical video with volumetric patient data for intraoperative navigation, which is incremental as it builds on existing Gaussian splatting and CT mesh techniques.
The paper tackles the problem of deformable surgical navigation by coupling intraoperative 3D reconstruction with preoperative CT data, resulting in a method that demonstrates sensible deformations of the CT on monocular RGB data in pig surgeries and synthetic human liver simulations.
We introduce BridgeSplat, a novel approach for deformable surgical navigation that couples intraoperative 3D reconstruction with preoperative CT data to bridge the gap between surgical video and volumetric patient data. Our method rigs 3D Gaussians to a CT mesh, enabling joint optimization of Gaussian parameters and mesh deformation through photometric supervision. By parametrizing each Gaussian relative to its parent mesh triangle, we enforce alignment between Gaussians and mesh and obtain deformations that can be propagated back to update the CT. We demonstrate BridgeSplat's effectiveness on visceral pig surgeries and synthetic data of a human liver under simulation, showing sensible deformations of the preoperative CT on monocular RGB data. Code, data, and additional resources can be found at https://maxfehrentz.github.io/ct-informed-splatting/ .