From Images2Mesh: A 3D Surface Reconstruction Pipeline for Non-Cooperative Space Objects
For space debris removal and on-orbit servicing, this pipeline enables 3D reconstruction from real, unconstrained inspection footage, addressing a practical bottleneck in existing methods that rely on synthetic data or known camera poses.
The paper presents a pipeline for neural implicit surface reconstruction of non-cooperative space objects from monocular inspection imagery, demonstrating it on real ISS and H-IIA rocket footage. They find that segmentation-based background removal is essential for camera pose estimation, and photometric correction helps but performance varies with illumination.
On-orbit inspection imagery is crucial as it enables characterization of non-cooperative resident space objects, providing the geometry and structural condition essential for active debris removal and on-orbit servicing mission planning. However, most existing neural implicit surface reconstruction methods have been confined to synthetic or hardware-in-the-loop data with known camera poses and controlled illumination. In this work, we present a pipeline for neural implicit surface reconstruction of non-cooperative space objects from monocular inspection imagery. We demonstrate it on publicly released ISS inspection footage from the STS-119 mission and publicly released on-orbit inspection footage of an H-IIA rocket upper stage. We find that segmentation-based background removal is essential for successful camera pose estimation from real on-orbit footage, where background variation between frames caused direct processing to fail entirely. We further incorporate photometric correction of per-frame exposure variations and analyze its behavior across datasets, finding that performance in shadowed regions varies with the illumination characteristics of the input footage.