WaveSSM: Multiscale State-Space Models for Non-stationary Signal Attention
This work addresses the challenge of handling transient dynamics in signals for applications like physiological monitoring and audio processing, representing an incremental improvement over prior SSM methods.
The paper tackled the problem of modeling non-stationary signals with localized or transient structures using state-space models, and found that WaveSSM outperforms existing methods like S4 on real-world datasets such as PTB-XL and Speech Commands.
State-space models (SSMs) have emerged as a powerful foundation for long-range sequence modeling, with the HiPPO framework showing that continuous-time projection operators can be used to derive stable, memory-efficient dynamical systems that encode the past history of the input signal. However, existing projection-based SSMs often rely on polynomial bases with global temporal support, whose inductive biases are poorly matched to signals exhibiting localized or transient structure. In this work, we introduce \emph{WaveSSM}, a collection of SSMs constructed over wavelet frames. Our key observation is that wavelet frames yield a localized support on the temporal dimension, useful for tasks requiring precise localization. Empirically, we show that on equal conditions, \textit{WaveSSM} outperforms orthogonal counterparts as S4 on real-world datasets with transient dynamics, including physiological signals on the PTB-XL dataset and raw audio on Speech Commands.