iSDF: Real-Time Neural Signed Distance Fields for Robot Perception
This work addresses the problem of real-time 3D environment reconstruction for robots in navigation and manipulation, representing an incremental improvement over prior voxel-based methods.
The paper tackles real-time signed distance field (SDF) reconstruction for robot perception by introducing iSDF, a continual learning system that uses a neural network to map 3D coordinates to signed distances, resulting in more accurate reconstructions and better approximations for downstream planning tasks compared to alternative methods.
We present iSDF, a continual learning system for real-time signed distance field (SDF) reconstruction. Given a stream of posed depth images from a moving camera, it trains a randomly initialised neural network to map input 3D coordinate to approximate signed distance. The model is self-supervised by minimising a loss that bounds the predicted signed distance using the distance to the closest sampled point in a batch of query points that are actively sampled. In contrast to prior work based on voxel grids, our neural method is able to provide adaptive levels of detail with plausible filling in of partially observed regions and denoising of observations, all while having a more compact representation. In evaluations against alternative methods on real and synthetic datasets of indoor environments, we find that iSDF produces more accurate reconstructions, and better approximations of collision costs and gradients useful for downstream planners in domains from navigation to manipulation. Code and video results can be found at our project page: https://joeaortiz.github.io/iSDF/ .