gvnn: Neural Network Library for Geometric Computer Vision
This library addresses the gap between geometric computer vision and deep learning for researchers and practitioners, though it is incremental as it builds on existing spatial transformer concepts.
The authors introduced gvnn, a Torch library that integrates geometric computer vision layers into neural networks, enabling end-to-end learning for tasks like place recognition, visual odometry, and depth estimation.
We introduce gvnn, a neural network library in Torch aimed towards bridging the gap between classic geometric computer vision and deep learning. Inspired by the recent success of Spatial Transformer Networks, we propose several new layers which are often used as parametric transformations on the data in geometric computer vision. These layers can be inserted within a neural network much in the spirit of the original spatial transformers and allow backpropagation to enable end-to-end learning of a network involving any domain knowledge in geometric computer vision. This opens up applications in learning invariance to 3D geometric transformation for place recognition, end-to-end visual odometry, depth estimation and unsupervised learning through warping with a parametric transformation for image reconstruction error.