Unconstrained Parametrization of Dissipative and Contracting Neural Ordinary Differential Equations
This work addresses robust learning and control in continuous-time neural networks, but it is incremental as it builds on existing Neural ODEs and RENs.
The authors tackled the problem of ensuring contractivity and dissipativity in Neural ODEs for robust learning and control by introducing NodeRENs, a class combining Neural ODEs with Recurrent Equilibrium Networks, and derived unconstrained parametrizations to enable learning with many parameters, validated in a nonlinear system identification case study.
In this work, we introduce and study a class of Deep Neural Networks (DNNs) in continuous-time. The proposed architecture stems from the combination of Neural Ordinary Differential Equations (Neural ODEs) with the model structure of recently introduced Recurrent Equilibrium Networks (RENs). We show how to endow our proposed NodeRENs with contractivity and dissipativity -- crucial properties for robust learning and control. Most importantly, as for RENs, we derive parametrizations of contractive and dissipative NodeRENs which are unconstrained, hence enabling their learning for a large number of parameters. We validate the properties of NodeRENs, including the possibility of handling irregularly sampled data, in a case study in nonlinear system identification.