SaFARi: State-Space Models for Frame-Agnostic Representation
This work addresses a bottleneck in SSM design for researchers and practitioners in machine learning, offering a more flexible foundation for modeling long-range dependent data, though it is incremental as it builds upon existing SSM concepts.
The authors tackled the limitation of State-Space Models (SSMs) being restricted to polynomial bases by introducing SaFARi, a generalized framework that allows SSMs to use any frame or basis, expanding beyond the HiPPO approach to enable diverse SSM architectures.
State-Space Models (SSMs) have re-emerged as a powerful tool for online function approximation, and as the backbone of machine learning models for long-range dependent data. However, to date, only a few polynomial bases have been explored for this purpose, and the state-of-the-art implementations were built upon the best of a few limited options. In this paper, we present a generalized method for building an SSM with any frame or basis, rather than being restricted to polynomials. This framework encompasses the approach known as HiPPO, but also permits an infinite diversity of other possible "species" within the SSM architecture. We dub this approach SaFARi: SSMs for Frame-Agnostic Representation.