GFreeDet: Exploiting Gaussian Splatting and Foundation Models for Model-free Unseen Object Detection in the BOP Challenge 2024
This enables robust model-free detection for mixed reality applications, addressing a bottleneck in handling novel objects.
The paper tackles the problem of unseen object detection without predefined CAD models by using Gaussian splatting and vision foundation models, achieving comparable performance to CAD-based methods on the BOP-H3 benchmark and winning awards in the BOP Challenge 2024.
We present GFreeDet, an unseen object detection approach that leverages Gaussian splatting and vision Foundation models under model-free setting. Unlike existing methods that rely on predefined CAD templates, GFreeDet reconstructs objects directly from reference videos using Gaussian splatting, enabling robust detection of novel objects without prior 3D models. Evaluated on the BOP-H3 benchmark, GFreeDet achieves comparable performance to CAD-based methods, demonstrating the viability of model-free detection for mixed reality (MR) applications. Notably, GFreeDet won the best overall method and the best fast method awards in the model-free 2D detection track at BOP Challenge 2024.