Iterative Linear Quadratic Optimization for Nonlinear Control: Differentiable Programming Algorithmic Templates
This work provides a unified implementation for control algorithms, which is incremental as it adapts existing methods to a new programming paradigm.
The paper tackles the problem of implementing nonlinear control algorithms within a differentiable programming framework, showing how methods like gradient descent and Newton can be unified and applied to benchmarks such as autonomous car racing simulations.
Iterative optimization algorithms depend on access to information about the objective function. In a differentiable programming framework, this information, such as gradients, can be automatically derived from the computational graph. We explore how nonlinear control algorithms, often employing linear and/or quadratic approximations, can be effectively cast within this framework. Our approach illuminates shared components and differences between gradient descent, Gauss-Newton, Newton, and differential dynamic programming methods in the context of discrete time nonlinear control. Furthermore, we present line-search strategies and regularized variants of these algorithms, along with a comprehensive analysis of their computational complexities. We study the performance of the aforementioned algorithms on various nonlinear control benchmarks, including autonomous car racing simulations using a simplified car model. All implementations are publicly available in a package coded in a differentiable programming language.