DM3D: Deformable Mamba via Offset-Guided Gaussian Sequencing for Point Cloud Understanding
This work solves the challenge of adapting SSMs for point cloud understanding, which is crucial for applications like 3D vision and robotics, representing a novel method for a known bottleneck rather than an incremental improvement.
The paper tackled the problem of applying State Space Models (SSMs) to point clouds by addressing the conflict between SSMs' reliance on input order and point clouds' irregular nature, proposing DM3D with an offset-guided Gaussian sequencing mechanism that achieves state-of-the-art performance in classification, few-shot learning, and part segmentation on benchmark datasets.
State Space Models (SSMs) demonstrate significant potential for long-sequence modeling, but their reliance on input order conflicts with the irregular nature of point clouds. Existing approaches often rely on predefined serialization strategies, which cannot adjust based on diverse geometric structures. To overcome this limitation, we propose \textbf{DM3D}, a deformable Mamba architecture for point cloud understanding. Specifically, DM3D introduces an offset-guided Gaussian sequencing mechanism that unifies local resampling and global reordering within a deformable scan. The Gaussian-based KNN Resampling (GKR) enhances structural awareness by adaptively reorganizing neighboring points, while the Gaussian-based Differentiable Reordering (GDR) enables end-to-end optimization of serialization order. Furthermore, a Tri-Path Frequency Fusion module enhances feature complementarity and reduces aliasing. Together, these components enable structure-adaptive serialization of point clouds. Extensive experiments on benchmark datasets show that DM3D achieves state-of-the-art performance in classification, few-shot learning, and part segmentation, demonstrating that adaptive serialization effectively unlocks the potential of SSMs for point cloud understanding.