SeanNet: Semantic Understanding Network for Localization Under Object Dynamics
This addresses the problem of long-term deployment for domestic robots in dynamic scenes, though it is incremental as it builds on existing visual localization methods.
The paper tackles robust robot localization in indoor environments with moving objects by proposing SeanNet, a semantic understanding network that uses coupled visual and semantic inputs and a cascaded contrastive learning scheme, achieving 85.02% accuracy in scene similarity measures and higher success rates in visual navigation compared to baselines.
We aim for domestic robots to perform long-term indoor service. Under the object-level scene dynamics induced by daily human activities, a robot needs to robustly localize itself in the environment subject to scene uncertainties. Previous works have addressed visual-based localization in static environments, yet the object-level scene dynamics challenge existing methods for the long-term deployment of the robot. This paper proposes a SEmantic understANding Network (SeanNet) architecture that enables an effective learning process with coupled visual and semantic inputs. With a dataset that contains object dynamics, we propose a cascaded contrastive learning scheme to train the SeanNet for learning a vector scene embedding. Subsequently, we can measure the similarity between the current observed scene and the target scene, whereby enables robust localization under object-level dynamics. In our experiments, we benchmark SeanNet against state-of-the-art image-encoding networks (baselines) on scene similarity measures. The SeanNet architecture with the proposed training method can achieve an 85.02\% accuracy which is higher than baselines. We further integrate the SeanNet and the other networks as the localizers into a visual navigation application. We demonstrate that SeanNet achieves higher success rates compared to the baselines.