Neural Time-Reversed Generalized Riccati Equation
This addresses the computational inefficiency of traditional Hamiltonian methods in optimal control, offering a more practical solution for applications in robotics, finance, and engineering, though it appears incremental as it builds on existing neural and LQ problem concepts.
The paper tackles the challenge of solving optimal control problems without relying on forward-backward algorithms by introducing a neural-based approach that works forward-in-time, using neural networks for state dynamics and costate estimation with a novel local policy called the time-reversed generalized Riccati equation, supported by experimental results from various case studies.
Optimal control deals with optimization problems in which variables steer a dynamical system, and its outcome contributes to the objective function. Two classical approaches to solving these problems are Dynamic Programming and the Pontryagin Maximum Principle. In both approaches, Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. However, Hamiltonian equations are rarely used due to their reliance on forward-backward algorithms across the entire temporal domain. This paper introduces a novel neural-based approach to optimal control, with the aim of working forward-in-time. Neural networks are employed not only for implementing state dynamics but also for estimating costate variables. The parameters of the latter network are determined at each time step using a newly introduced local policy referred to as the time-reversed generalized Riccati equation. This policy is inspired by a result discussed in the Linear Quadratic (LQ) problem, which we conjecture stabilizes state dynamics. We support this conjecture by discussing experimental results from a range of optimal control case studies.