Lightning Does Not Strike Twice: Robust MDPs with Coupled Uncertainty
This addresses the issue of robust decision-making in uncertain environments for applications like robotics or finance, offering a less conservative approach than previous uncoupled uncertainty models.
The paper tackles the problem of overly conservative solutions in Markov decision processes under parameter uncertainty by introducing a 'Lightning Does Not Strike Twice' concept to model coupled uncertainties, where deviations from nominal parameters are bounded, and provides tractable algorithms with probabilistic guarantees for optimal control policies.
We consider Markov decision processes under parameter uncertainty. Previous studies all restrict to the case that uncertainties among different states are uncoupled, which leads to conservative solutions. In contrast, we introduce an intuitive concept, termed "Lightning Does not Strike Twice," to model coupled uncertain parameters. Specifically, we require that the system can deviate from its nominal parameters only a bounded number of times. We give probabilistic guarantees indicating that this model represents real life situations and devise tractable algorithms for computing optimal control policies using this concept.