Hierarchical Neural Architecture Search for Deep Stereo Matching
This work addresses the problem of automating network design for low-level geometric vision tasks like stereo matching, offering a novel approach that outperforms existing methods, though it is incremental in applying NAS to a new domain.
The paper tackles the challenge of applying Neural Architecture Search (NAS) to deep stereo matching, which was previously computationally prohibitive due to large network sizes, by proposing a hierarchical NAS framework that incorporates task-specific knowledge to jointly optimize the entire pipeline. The result is a network that achieves top-1 accuracy on KITTI 2012, 2015, Middlebury, and SceneFlow benchmarks, with substantial improvements in network size and inference speed.
To reduce the human efforts in neural network design, Neural Architecture Search (NAS) has been applied with remarkable success to various high-level vision tasks such as classification and semantic segmentation. The underlying idea for the NAS algorithm is straightforward, namely, to enable the network the ability to choose among a set of operations (e.g., convolution with different filter sizes), one is able to find an optimal architecture that is better adapted to the problem at hand. However, so far the success of NAS has not been enjoyed by low-level geometric vision tasks such as stereo matching. This is partly due to the fact that state-of-the-art deep stereo matching networks, designed by humans, are already sheer in size. Directly applying the NAS to such massive structures is computationally prohibitive based on the currently available mainstream computing resources. In this paper, we propose the first end-to-end hierarchical NAS framework for deep stereo matching by incorporating task-specific human knowledge into the neural architecture search framework. Specifically, following the gold standard pipeline for deep stereo matching (i.e., feature extraction -- feature volume construction and dense matching), we optimize the architectures of the entire pipeline jointly. Extensive experiments show that our searched network outperforms all state-of-the-art deep stereo matching architectures and is ranked at the top 1 accuracy on KITTI stereo 2012, 2015 and Middlebury benchmarks, as well as the top 1 on SceneFlow dataset with a substantial improvement on the size of the network and the speed of inference. The code is available at https://github.com/XuelianCheng/LEAStereo.