Resilient Strategies for Stochastic Systems: How Much Does It Take to Break a Winning Strategy?
This work addresses the challenge of decision-making under uncertainty for agents in domains like robotics or control systems, but it appears incremental as it extends existing concepts to a stochastic setting.
The paper tackles the problem of designing resilient strategies for agents in stochastic systems, such as Markov decision processes, to ensure robustness against disturbances that can flip decisions, and it introduces a framework with various aggregation methods like expectation and worst-case analysis to quantify disturbances.
We study the problem of resilient strategies in the presence of uncertainty. Resilient strategies enable an agent to make decisions that are robust against disturbances. In particular, we are interested in those disturbances that are able to flip a decision made by the agent. Such a disturbance may, for instance, occur when the intended action of the agent cannot be executed due to a malfunction of an actuator in the environment. In this work, we introduce the concept of resilience in the stochastic setting and present a comprehensive set of fundamental problems. Specifically, we discuss such problems for Markov decision processes with reachability and safety objectives, which also smoothly extend to stochastic games. To account for the stochastic setting, we provide various ways of aggregating the amounts of disturbances that may have occurred, for instance, in expectation or in the worst case. Moreover, to reason about infinite disturbances, we use quantitative measures, like their frequency of occurrence.