Weakly chained matrices, policy iteration, and impulse control
Provides guidance for practitioners solving impulse control problems numerically, but the recommendation is incremental.
This work analyzes numerical schemes for Hamilton-Jacobi-Bellman quasi-variational inequalities arising from combined stochastic and impulse control problems. It finds that the direct control scheme should be avoided due to convergence issues, while penalized and semi-Lagrangian schemes are preferable.
This work is motivated by numerical solutions to Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) associated with combined stochastic and impulse control problems. In particular, we consider (i) direct control, (ii) penalized, and (iii) semi-Lagrangian discretization schemes applied to the HJBQVI problem. Scheme (i) takes the form of a Bellman problem involving an operator which is not necessarily contractive. We consider the well-posedness of the Bellman problem and give sufficient conditions for convergence of the corresponding policy iteration. To do so, we use weakly chained diagonally dominant matrices, which give a graph-theoretic characterization of weakly diagonally dominant M-matrices. We compare schemes (i)--(iii) under the following examples: (a) optimal control of the exchange rate, (b) optimal consumption with fixed and proportional transaction costs, and (c) pricing guaranteed minimum withdrawal benefits in variable annuities. We find that one should abstain from using scheme (i).