FoundationPose-Initialized 3D-2D Liver Registration for Surgical Augmented Reality
This work addresses the need for more accessible and less complex registration methods in surgical augmented reality for liver surgery, offering a clinically relevant and engineering-friendly alternative to existing finite-element-based approaches.
The paper tackled the problem of improving tumor localization in laparoscopic liver surgery by developing a registration pipeline that integrates laparoscopic depth maps with a foundation pose estimator and uses non-rigid iterative closest point (NICP) instead of finite-element models, achieving a mean registration error of 9.91 mm on real patient data.
Augmented reality can improve tumor localization in laparoscopic liver surgery. Existing registration pipelines typically depend on organ contours; deformable (non-rigid) alignment is often handled with finite-element (FE) models coupled to dimensionality-reduction or machine-learning components. We integrate laparoscopic depth maps with a foundation pose estimator for camera-liver pose estimation and replace FE-based deformation with non-rigid iterative closest point (NICP) to lower engineering/modeling complexity and expertise requirements. On real patient data, the depth-augmented foundation pose approach achieved 9.91 mm mean registration error in 3 cases. Combined rigid-NICP registration outperformed rigid-only registration, demonstrating NICP as an efficient substitute for finite-element deformable models. This pipeline achieves clinically relevant accuracy while offering a lightweight, engineering-friendly alternative to FE-based deformation.