DensePercept-NCSSD: Vision Mamba towards Real-time Dense Visual Perception with Non-Causal State Space Duality
This work addresses the need for efficient and accurate 3D perception in real-time applications, such as robotics or autonomous systems, though it appears incremental as it builds on existing Mamba and state space methods.
The paper tackles the problem of real-time dense visual perception, specifically optical flow and disparity estimation, by proposing a non-causal Mamba block-based model that reduces inference times while maintaining high accuracy and low GPU usage.
In this work, we propose an accurate and real-time optical flow and disparity estimation model by fusing pairwise input images in the proposed non-causal selective state space for dense perception tasks. We propose a non-causal Mamba block-based model that is fast and efficient and aptly manages the constraints present in a real-time applications. Our proposed model reduces inference times while maintaining high accuracy and low GPU usage for optical flow and disparity map generation. The results and analysis, and validation in real-life scenario justify that our proposed model can be used for unified real-time and accurate 3D dense perception estimation tasks. The code, along with the models, can be found at https://github.com/vimstereo/DensePerceptNCSSD