TR3D: Towards Real-Time Indoor 3D Object Detection
This work addresses the need for efficient and accurate 3D object detection in indoor environments, representing an incremental advancement by refining existing sparse 3D convolution methods.
The paper tackles the problem of improving 3D object detection for indoor scenes by developing TR3D, a fast fully-convolutional model that achieves state-of-the-art results on benchmarks like ScanNet v2, SUN RGB-D, and S3DIS, and introduces an early fusion module (TR3D+FF) that boosts performance by combining point cloud and RGB inputs.
Recently, sparse 3D convolutions have changed 3D object detection. Performing on par with the voting-based approaches, 3D CNNs are memory-efficient and scale to large scenes better. However, there is still room for improvement. With a conscious, practice-oriented approach to problem-solving, we analyze the performance of such methods and localize the weaknesses. Applying modifications that resolve the found issues one by one, we end up with TR3D: a fast fully-convolutional 3D object detection model trained end-to-end, that achieves state-of-the-art results on the standard benchmarks, ScanNet v2, SUN RGB-D, and S3DIS. Moreover, to take advantage of both point cloud and RGB inputs, we introduce an early fusion of 2D and 3D features. We employ our fusion module to make conventional 3D object detection methods multimodal and demonstrate an impressive boost in performance. Our model with early feature fusion, which we refer to as TR3D+FF, outperforms existing 3D object detection approaches on the SUN RGB-D dataset. Overall, besides being accurate, both TR3D and TR3D+FF models are lightweight, memory-efficient, and fast, thereby marking another milestone on the way toward real-time 3D object detection. Code is available at https://github.com/SamsungLabs/tr3d .