Supervised Learning for Dynamical System Learning
This work addresses a practical limitation for researchers and practitioners using spectral methods in dynamical system learning, offering a more flexible and extensible approach, though it is incremental as it builds on existing spectral methods.
The paper tackles the difficulty of incorporating prior information like sparsity into spectral methods for learning dynamical systems by proposing a new framework that reformulates the problem as a sequence of supervised learning tasks, enabling the use of standard techniques such as L1 regularization. It demonstrates that nonlinear regression or lasso can learn better state representations than linear regression, with correctness proven through general analysis.
Recently there has been substantial interest in spectral methods for learning dynamical systems. These methods are popular since they often offer a good tradeoff between computational and statistical efficiency. Unfortunately, they can be difficult to use and extend in practice: e.g., they can make it difficult to incorporate prior information such as sparsity or structure. To address this problem, we present a new view of dynamical system learning: we show how to learn dynamical systems by solving a sequence of ordinary supervised learning problems, thereby allowing users to incorporate prior knowledge via standard techniques such as L1 regularization. Many existing spectral methods are special cases of this new framework, using linear regression as the supervised learner. We demonstrate the effectiveness of our framework by showing examples where nonlinear regression or lasso let us learn better state representations than plain linear regression does; the correctness of these instances follows directly from our general analysis.