Attention Mamba: Time Series Modeling with Adaptive Pooling Acceleration and Receptive Field Enhancements
This work addresses time series modeling problems for applications like weather forecasting and transportation management, presenting an incremental improvement over existing Mamba-based approaches.
The paper tackles challenges in time series modeling, such as insufficient nonlinear dependency modeling and restricted receptive fields in existing methods like Mamba, by introducing Attention Mamba with an Adaptive Pooling block and bidirectional Mamba block, achieving superior performance on diverse datasets.
"This work has been submitted to the lEEE for possible publication. Copyright may be transferred without noticeafter which this version may no longer be accessible." Time series modeling serves as the cornerstone of real-world applications, such as weather forecasting and transportation management. Recently, Mamba has become a promising model that combines near-linear computational complexity with high prediction accuracy in time series modeling, while facing challenges such as insufficient modeling of nonlinear dependencies in attention and restricted receptive fields caused by convolutions. To overcome these limitations, this paper introduces an innovative framework, Attention Mamba, featuring a novel Adaptive Pooling block that accelerates attention computation and incorporates global information, effectively overcoming the constraints of limited receptive fields. Furthermore, Attention Mamba integrates a bidirectional Mamba block, efficiently capturing long-short features and transforming inputs into the Value representations for attention mechanisms. Extensive experiments conducted on diverse datasets underscore the effectiveness of Attention Mamba in extracting nonlinear dependencies and enhancing receptive fields, establishing superior performance among leading counterparts. Our codes will be available on GitHub.