Multi-scale Alternated Attention Transformer for Generalized Stereo Matching
This work addresses stereo matching for computer vision applications, offering incremental improvements in generalization performance over existing methods.
The paper tackles the problem of stereo matching in complex real-world scenarios by proposing the Alternated Attention U-shaped Transformer (AAUformer), which balances epipolar line constraints across dual and single views to improve generalization. The model achieves state-of-the-art results on the Scene Flow dataset and competitive performance on KITTI 2015, with strong cross-generalization between synthetic and real-world datasets.
Recent stereo matching networks achieves dramatic performance by introducing epipolar line constraint to limit the matching range of dual-view. However, in complicated real-world scenarios, the feature information based on intra-epipolar line alone is too weak to facilitate stereo matching. In this paper, we present a simple but highly effective network called Alternated Attention U-shaped Transformer (AAUformer) to balance the impact of epipolar line in dual and single view respectively for excellent generalization performance. Compared to other models, our model has several main designs: 1) to better liberate the local semantic features of the single-view at pixel level, we introduce window self-attention to break the limits of intra-row self-attention and completely replace the convolutional network for denser features before cross-matching; 2) the multi-scale alternated attention backbone network was designed to extract invariant features in order to achieves the coarse-to-fine matching process for hard-to-discriminate regions. We performed a series of both comparative studies and ablation studies on several mainstream stereo matching datasets. The results demonstrate that our model achieves state-of-the-art on the Scene Flow dataset, and the fine-tuning performance is competitive on the KITTI 2015 dataset. In addition, for cross generalization experiments on synthetic and real-world datasets, our model outperforms several state-of-the-art works.