Depth-Adapted CNN for RGB-D cameras
This work addresses the challenge of integrating geometric information into CNNs for RGB-D camera applications, offering a novel approach that could enhance performance in tasks like object recognition or scene understanding, though it appears incremental relative to existing depth-based methods.
The paper tackles the problem of improving RGB CNN methods by incorporating depth information from RGB-D cameras, proposing a depth-adapted CNN that uses depth data as 2D offsets for spatial sampling, achieving invariance to scale and rotation in camera coordinates and validating effectiveness through benchmark experiments.
Conventional 2D Convolutional Neural Networks (CNN) extract features from an input image by applying linear filters. These filters compute the spatial coherence by weighting the photometric information on a fixed neighborhood without taking into account the geometric information. We tackle the problem of improving the classical RGB CNN methods by using the depth information provided by the RGB-D cameras. State-of-the-art approaches use depth as an additional channel or image (HHA) or pass from 2D CNN to 3D CNN. This paper proposes a novel and generic procedure to articulate both photometric and geometric information in CNN architecture. The depth data is represented as a 2D offset to adapt spatial sampling locations. The new model presented is invariant to scale and rotation around the X and the Y axis of the camera coordinate system. Moreover, when depth data is constant, our model is equivalent to a regular CNN. Experiments of benchmarks validate the effectiveness of our model.