A Local Information Criterion for Dynamical Systems
This addresses the challenge of selecting accurate models for dynamical systems in applications like data compression and learning, but it appears incremental as it builds on existing encoding methods for model selection.
The paper tackles the problem of encoding sequences generated by dynamical systems and proposes using the compression efficiency of this encoding as a model selection criterion to choose models closer to the true underlying dynamics, with experiments and theoretical results provided.
Encoding a sequence of observations is an essential task with many applications. The encoding can become highly efficient when the observations are generated by a dynamical system. A dynamical system imposes regularities on the observations that can be leveraged to achieve a more efficient code. We propose a method to encode a given or learned dynamical system. Apart from its application for encoding a sequence of observations, we propose to use the compression achieved by this encoding as a criterion for model selection. Given a dataset, different learning algorithms result in different models. But not all learned models are equally good. We show that the proposed encoding approach can be used to choose the learned model which is closer to the true underlying dynamics. We provide experiments for both encoding and model selection, and theoretical results that shed light on why the approach works.