Learnable & Interpretable Model Combination in Dynamical Systems Modeling
This work addresses the challenge of creating more powerful or efficient models for dynamical systems applications, but it appears incremental as it builds on existing combination methods and focuses on specific theoretical issues.
The paper tackles the problem of combining multiple model architectures in dynamical systems modeling, proposing a new wildcard architecture that can describe arbitrary combinations in an interpretable and learnable way, with experiments demonstrating its application in learning and comparing different combination architectures.
During modeling of dynamical systems, often two or more model architectures are combined to obtain a more powerful or efficient model regarding a specific application area. This covers the combination of multiple machine learning architectures, as well as hybrid models, i.e., the combination of physical simulation models and machine learning. In this work, we briefly discuss which types of model are usually combined in dynamical systems modeling and propose a class of models that is capable of expressing mixed algebraic, discrete, and differential equation-based models. Further, we examine different established, as well as new ways of combining these models from the point of view of system theory and highlight two challenges - algebraic loops and local event functions in discontinuous models - that require a special approach. Finally, we propose a new wildcard architecture that is capable of describing arbitrary combinations of models in an easy-to-interpret fashion that can be learned as part of a gradient-based optimization procedure. In a final experiment, different combination architectures between two models are learned, interpreted, and compared using the methodology and software implementation provided.