Opportunistic Qualitative Planning in Stochastic Systems with Incomplete Preferences over Reachability Objectives
This work addresses the challenge of decision-making under uncertainty with qualitative preferences, which is incremental as it builds on existing MDP frameworks by incorporating novel preference semantics and improvement notions.
The paper tackles the problem of synthesizing plans that satisfy incomplete preferences over reachability objectives in stochastic systems modeled as Markov Decision Processes (MDPs), by introducing new semantics for preferences over infinite plays and solution concepts (SPI and SASI) that enforce improvements with positive probability or probability one, and demonstrates this approach in a robot motion planning problem.
Preferences play a key role in determining what goals/constraints to satisfy when not all constraints can be satisfied simultaneously. In this paper, we study how to synthesize preference satisfying plans in stochastic systems, modeled as an MDP, given a (possibly incomplete) combinative preference model over temporally extended goals. We start by introducing new semantics to interpret preferences over infinite plays of the stochastic system. Then, we introduce a new notion of improvement to enable comparison between two prefixes of an infinite play. Based on this, we define two solution concepts called safe and positively improving (SPI) and safe and almost-surely improving (SASI) that enforce improvements with a positive probability and with probability one, respectively. We construct a model called an improvement MDP, in which the synthesis of SPI and SASI strategies that guarantee at least one improvement reduces to computing positive and almost-sure winning strategies in an MDP. We present an algorithm to synthesize the SPI and SASI strategies that induce multiple sequential improvements. We demonstrate the proposed approach using a robot motion planning problem.