Approximate Dynamic Programming For Linear Systems with State and Input Constraints
This addresses a crucial roadblock for using RL in safety-critical applications, though it appears incremental as it builds on existing ADP and invariant set methods.
The paper tackled the problem of enforcing state and input constraints in reinforcement learning for continuous systems, proposing an approximate dynamic programming framework that guarantees constraint satisfaction and converges to the optimal policy asymptotically, as demonstrated through numerical examples.
Enforcing state and input constraints during reinforcement learning (RL) in continuous state spaces is an open but crucial problem which remains a roadblock to using RL in safety-critical applications. This paper leverages invariant sets to update control policies within an approximate dynamic programming (ADP) framework that guarantees constraint satisfaction for all time and converges to the optimal policy (in a linear quadratic regulator sense) asymptotically. An algorithm for implementing the proposed constrained ADP approach in a data-driven manner is provided. The potential of this formalism is demonstrated via numerical examples.