Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach
For control practitioners facing data-driven uncertainty, this work provides a theoretically grounded and computationally feasible robust control method.
This paper addresses stochastic control under distributional uncertainty by proposing a Wasserstein distributionally robust framework. It develops tractable value and policy iteration algorithms with explicit iteration bounds for ε-optimal policies, and extends single-stage out-of-sample guarantees to multi-stage without confidence degradation.
Standard stochastic control methods assume that the probability distribution of uncertain variables is available. Unfortunately, in practice, obtaining accurate distribution information is a challenging task. To resolve this issue, we investigate the problem of designing a control policy that is robust against errors in the empirical distribution obtained from data. This problem can be formulated as a two-player zero-sum dynamic game problem, where the action space of the adversarial player is a Wasserstein ball centered at the empirical distribution. We propose computationally tractable value and policy iteration algorithms with explicit estimates of the number of iterations required for constructing an $ε$-optimal policy. We show that the contraction property of associated Bellman operators extends a single-stage out-of-sample performance guarantee, obtained using a measure concentration inequality, to the corresponding multi-stage guarantee without any degradation in the confidence level. In addition, we characterize an explicit form of the optimal distributionally robust control policy and the worst-case distribution policy for linear-quadratic problems with Wasserstein penalty. Our study indicates that dynamic programming and Kantorovich duality play a critical role in solving and analyzing the Wasserstein distributionally robust stochastic control problems.