NeRF-based Visualization of 3D Cues Supporting Data-Driven Spacecraft Pose Estimation
This addresses the lack of interpretability in spacecraft pose estimation for on-orbit operations, which is an incremental improvement in visualization techniques.
The paper tackles the problem of understanding the decision process in data-driven spacecraft pose estimation by presenting a method to visualize the 3D visual cues relied upon by the estimator, using a NeRF-based image generator trained with gradients from the pose network, and experiments show it recovers relevant 3D cues and provides insights into the network's implicit representation.
On-orbit operations require the estimation of the relative 6D pose, i.e., position and orientation, between a chaser spacecraft and its target. While data-driven spacecraft pose estimation methods have been developed, their adoption in real missions is hampered by the lack of understanding of their decision process. This paper presents a method to visualize the 3D visual cues on which a given pose estimator relies. For this purpose, we train a NeRF-based image generator using the gradients back-propagated through the pose estimation network. This enforces the generator to render the main 3D features exploited by the spacecraft pose estimation network. Experiments demonstrate that our method recovers the relevant 3D cues. Furthermore, they offer additional insights on the relationship between the pose estimation network supervision and its implicit representation of the target spacecraft.