Evaluation of the Rate of Convergence in the PIA
For researchers using policy improvement algorithms in controlled diffusion or PDE contexts, this provides theoretical justification for observed fast convergence, though it is an incremental extension of prior work.
The paper proves quadratic local convergence of Howard's Policy Improvement Algorithm in a general setup, explaining its fast convergence, and demonstrates this with a numerical example solving a semilinear elliptic PDE.
Folklore says that Howard's Policy Improvement Algorithm converges extraordinarily fast, even for controlled diffusion settings. In a previous paper, we proved that approximations of the solution of a particular parabolic partial differential equation obtained via the policy improvement algorithm show a quadratic local convergence. In this paper, we show that we obtain the same rate of convergence of the algorithm in a more general setup. This provides some explanation as to why the algorithm converges fast. We provide an example by solving a semilinear elliptic partial differential equation numerically by applying the algorithm and check how the approximations converge to the analytic solution.