Leveraging Neural Radiance Fields for Pose Estimation of an Unknown Space Object during Proximity Operations
This addresses the challenge of autonomous spacecraft rendezvous for debris removal missions, but it is incremental as it adapts existing methods to new conditions.
The paper tackles the problem of estimating the 6D pose of an unknown spacecraft using a monocular camera, which is crucial for autonomous operations like debris removal, by enabling an off-the-shelf pose estimator to work without a CAD model through a NeRF-based method that generates diverse training data from sparse images, achieving performance similar to models trained on synthetic CAD data.
We address the estimation of the 6D pose of an unknown target spacecraft relative to a monocular camera, a key step towards the autonomous rendezvous and proximity operations required by future Active Debris Removal missions. We present a novel method that enables an "off-the-shelf" spacecraft pose estimator, which is supposed to known the target CAD model, to be applied on an unknown target. Our method relies on an in-the wild NeRF, i.e., a Neural Radiance Field that employs learnable appearance embeddings to represent varying illumination conditions found in natural scenes. We train the NeRF model using a sparse collection of images that depict the target, and in turn generate a large dataset that is diverse both in terms of viewpoint and illumination. This dataset is then used to train the pose estimation network. We validate our method on the Hardware-In-the-Loop images of SPEED+ that emulate lighting conditions close to those encountered on orbit. We demonstrate that our method successfully enables the training of an off-the-shelf spacecraft pose estimation network from a sparse set of images. Furthermore, we show that a network trained using our method performs similarly to a model trained on synthetic images generated using the CAD model of the target.