Optimizing Hard Thresholding for Sparse Model Discovery
This incremental improvement addresses sparse dictionary learning for interpretable physics-based models, benefiting researchers in fields like fluid dynamics and population dynamics.
The paper tackled the problem of improving sparse model selection algorithms by introducing an annealing scheme that reactivates removed terms, which enhanced model accuracy in various nonlinear systems and experimental data.
Many model selection algorithms rely on sparse dictionary learning to provide interpretable and physics-based governing equations. The optimization algorithms typically use a hard thresholding process to enforce sparse activations in the model coefficients by removing library elements from consideration. By introducing an annealing scheme that reactivates a fraction of the removed terms with a cooling schedule, we are able to improve the performance of these sparse learning algorithms. We concentrate on two approaches to the optimization, SINDy, and an alternative using hard thresholding pursuit. We see in both cases that annealing can improve model accuracy. The effectiveness of annealing is demonstrated through comparisons on several nonlinear systems pulled from convective flows, excitable systems, and population dynamics. Finally we apply these algorithms to experimental data for projectile motion.