Statistical Mechanics of Dynamical System Identification
This work addresses the challenge of system identification for researchers in fields like physics and engineering, offering a novel theoretical approach with potential for broad adaptation, though it is incremental in applying existing statistical mechanics concepts to this domain.
The paper tackles the problem of recovering dynamical equations from noisy data by developing a statistical mechanics framework to analyze sparse equation discovery algorithms, providing uncertainty quantification and characterizing phase transitions that delineate correct from incorrect identification.
Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanics approach to analyze sparse equation discovery algorithms, which typically balance data fit and parsimony via hyperparameter tuning. In this framework, statistical mechanics offers tools to analyze the interplay between complexity and fitness similarly to that of entropy and energy in physical systems. To establish this analogy, we define the hyperparameter optimization procedure as a two-level Bayesian inference problem that separates variable selection from coefficient inference and enables the computation of the posterior parameter distribution in closed form. Our approach provides uncertainty quantification, crucial in the low-data limit that is frequently encountered in real-world applications. A key advantage of employing statistical mechanical concepts, such as free energy and the partition function, is to connect the large data limit to thermodynamic limit and characterize the sparsity- and noise-induced phase transitions that delineate correct from incorrect identification. We thus provide a method for closed-loop inference, estimating the noise in a given model and checking if the model is tolerant to that noise amount. This perspective of sparse equation discovery is versatile and can be adapted to various other equation discovery algorithms.