Orthogonal projection-based regularization for efficient model augmentation
This work addresses the problem of integrating prior physical knowledge into deep-learning models for nonlinear system identification, which is incremental as it builds on existing additive augmentation approaches.
The paper tackles the challenge of overparameterization and training difficulty in additive model augmentation structures that combine physics-based and machine-learning components, proposing an orthogonal projection-based regularization technique to improve parameter learning and model accuracy.
Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. However, these black-box models lack physical interpretability, and a considerable part of the learning effort is often spent on capturing already expected/known behavior of the system, that can be accurately described by first-principles laws of physics. A potential solution is to directly integrate such prior physical knowledge into the model structure, combining the strengths of physics-based modeling and deep-learning-based identification. The most common approach is to use an additive model augmentation structure, where the physics-based and the machine-learning (ML) components are connected in parallel, i.e., additively. However, such models are overparametrized, training them is challenging, potentially causing the physics-based part to lose interpretability. To overcome this challenge, this paper proposes an orthogonal projection-based regularization technique to enhance parameter learning and even model accuracy in learning-based augmentation of nonlinear baseline models.