MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps
This work addresses the problem of precise 3D object detection in indoor scenes for applications like robotics and augmented reality, representing an incremental improvement over previous methods.
The paper tackles the challenge of inaccurate geometry in multi-view indoor 3D object detection by proposing MVSDet, which uses plane sweeps with probabilistic sampling and soft weighting, achieving state-of-the-art results on ScanNet and ARKitScenes datasets.
The key challenge of multi-view indoor 3D object detection is to infer accurate geometry information from images for precise 3D detection. Previous method relies on NeRF for geometry reasoning. However, the geometry extracted from NeRF is generally inaccurate, which leads to sub-optimal detection performance. In this paper, we propose MVSDet which utilizes plane sweep for geometry-aware 3D object detection. To circumvent the requirement for a large number of depth planes for accurate depth prediction, we design a probabilistic sampling and soft weighting mechanism to decide the placement of pixel features on the 3D volume. We select multiple locations that score top in the probability volume for each pixel and use their probability score to indicate the confidence. We further apply recent pixel-aligned Gaussian Splatting to regularize depth prediction and improve detection performance with little computation overhead. Extensive experiments on ScanNet and ARKitScenes datasets are conducted to show the superiority of our model. Our code is available at https://github.com/Pixie8888/MVSDet.