Differentiable Implicit Layers
This addresses the challenge of integrating implicit layers into neural networks for researchers and practitioners in machine learning and control systems, representing an incremental improvement in optimization methods.
The paper tackles the problem of efficiently training neural networks with non-constrained implicit functions as layers by introducing a new backpropagation scheme, demonstrating its application to neural ODEs with implicit Euler and system identification in model predictive control.
In this paper, we introduce an efficient backpropagation scheme for non-constrained implicit functions. These functions are parametrized by a set of learnable weights and may optionally depend on some input; making them perfectly suitable as a learnable layer in a neural network. We demonstrate our scheme on different applications: (i) neural ODEs with the implicit Euler method, and (ii) system identification in model predictive control.