The Cesàro Value Iteration
This work provides a novel solution for a known bottleneck in optimal control for systems where standard value iteration fails, though limited to periodic optimal behavior.
The paper addresses the non-convergence of classic value iteration in undiscounted infinite-horizon optimal control for deterministic systems with uncountable state and input spaces. It introduces Cesàro value iteration, proving its convergence for systems with periodic optimal behavior and showing it recovers the optimal cost when defined.
In this paper, we consider undiscouted infinite-horizon optimal control for deterministic systems with an uncountable state and input space. We specifically address the case when the classic value iteration does not converge. For such systems, we use the Ces`aro mean to define the infinite-horizon optimal control problem and the corresponding infinite-horizon value function. Moreover, for this value function, we introduce the Cesàro value iteration and prove its convergence for the special case of systems with periodic optimal operating behavior. For this instance, we also show that the Cesàro value function recovers the undiscounted infinite-horizon optimal cost, if the latter is well-defined.