Optimizing Quantiles in Preference-based Markov Decision Processes
This work addresses the need for alternative decision criteria in MDPs, particularly for scenarios where expected rewards are insufficient, though it appears incremental as it adapts existing MDP frameworks to a quantile-based approach.
The paper tackles the problem of evaluating policies in Markov decision processes using quantiles instead of expected cumulative rewards, proposing an algorithm for computing optimal policies under this criterion and demonstrating its performance on random MDPs and a data center control problem.
In the Markov decision process model, policies are usually evaluated by expected cumulative rewards. As this decision criterion is not always suitable, we propose in this paper an algorithm for computing a policy optimal for the quantile criterion. Both finite and infinite horizons are considered. Finally we experimentally evaluate our approach on random MDPs and on a data center control problem.