RBF Weighted Hyper-Involution for RGB-D Object Detection
This addresses the problem of efficient object detection for augmented reality devices by improving real-time processing of depth and color images, though it appears incremental with novel components.
The paper tackles the challenge of real-time RGB-D object detection by proposing a two-stream model with a depth-guided hyper-involution and a trainable fusion layer, achieving state-of-the-art results on NYU Depth v2 and competitive performance on SUN RGB-D, with a new outdoor dataset introduced.
A vast majority of conventional augmented reality devices are equipped with depth sensors. Depth images produced by such sensors contain complementary information for object detection when used with color images. Despite the benefits, it remains a complex task to simultaneously extract photometric and depth features in real time due to the immanent difference between depth and color images. Moreover, standard convolution operations are not sufficient to properly extract information directly from raw depth images leading to intermediate representations of depth which is inefficient. To address these issues, we propose a real-time and two stream RGBD object detection model. The proposed model consists of two new components: a depth guided hyper-involution that adapts dynamically based on the spatial interaction pattern in the raw depth map and an up-sampling based trainable fusion layer that combines the extracted depth and color image features without blocking the information transfer between them. We show that the proposed model outperforms other RGB-D based object detection models on NYU Depth v2 dataset and achieves comparable (second best) results on SUN RGB-D. Additionally, we introduce a new outdoor RGB-D object detection dataset where our proposed model outperforms other models. The performance evaluation on diverse synthetic data generated from CAD models and images shows the potential of the proposed model to be adapted to augmented reality based applications.