Open-Set Semantic Uncertainty Aware Metric-Semantic Graph Matching
This work addresses the challenge of handling uncommon and unseen object classes in underwater robotics, offering an incremental improvement in loop closure detection for marine and terrestrial applications.
The paper tackles the problem of robust object-level loop closure detection for unknown object classes in underwater mapping by incorporating a metric of semantic uncertainty from visual foundation models into a graph matching framework, achieving feasibility for real-time use in marine environments and generalization to terrestrial scenes like KITTI.
Underwater object-level mapping requires incorporating visual foundation models to handle the uncommon and often previously unseen object classes encountered in marine scenarios. In this work, a metric of semantic uncertainty for open-set object detections produced by visual foundation models is calculated and then incorporated into an object-level uncertainty tracking framework. Object-level uncertainties and geometric relationships between objects are used to enable robust object-level loop closure detection for unknown object classes. The above loop closure detection problem is formulated as a graph-matching problem. While graph matching, in general, is NP-Complete, a solver for an equivalent formulation of the proposed graph matching problem as a graph editing problem is tested on multiple challenging underwater scenes. Results for this solver as well as three other solvers demonstrate that the proposed methods are feasible for real-time use in marine environments for the robust, open-set, multi-object, semantic-uncertainty-aware loop closure detection. Further experimental results on the KITTI dataset demonstrate that the method generalizes to large-scale terrestrial scenes.