Cascade Markov Decision Processes: Theory and Applications
Provides a theoretical framework for a new class of MDPs, but the results are preliminary and limited to specific cases, making it an incremental contribution for researchers in stochastic control.
The paper introduces cascade Markov decision processes (MDPs) for controlling time-varying continuous-time Markov chains with Markovian transition rates, deriving optimal performance via ODEs in some cases and identifying singular control problems in others, with applications in finance and behavioral decision making.
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine the optimal performance and feedback controls, while some other cases are shown to lead to singular optimal control problems which are more difficult to solve. Solution techniques are demonstrated using examples from finance to behavioral decision making.