Stochastic Shortest Path with Adversarially Changing Costs
This addresses a natural setting in planning and control for agents operating in dynamic environments, but it is incremental as it extends existing SSP models to include adversarial cost changes.
The paper tackles the problem of stochastic shortest path with adversarially changing costs, where an agent must reach a goal with minimal expected cost despite unknown transitions and arbitrarily varying costs over episodes, and achieves sub-linear regret bounds of $\widetilde O (\sqrt{K})$ for positive costs and $\widetilde O (K^{3/4})$ in general.
Stochastic shortest path (SSP) is a well-known problem in planning and control, in which an agent has to reach a goal state in minimum total expected cost. In this paper we present the adversarial SSP model that also accounts for adversarial changes in the costs over time, while the underlying transition function remains unchanged. Formally, an agent interacts with an SSP environment for $K$ episodes, the cost function changes arbitrarily between episodes, and the transitions are unknown to the agent. We develop the first algorithms for adversarial SSPs and prove high probability regret bounds of $\widetilde O (\sqrt{K})$ assuming all costs are strictly positive, and $\widetilde O (K^{3/4})$ in the general case. We are the first to consider this natural setting of adversarial SSP and obtain sub-linear regret for it.