Multi-Target Decision Making under Conditions of Severe Uncertainty
This work addresses decision-making under uncertainty for fields like algorithm evaluation, though it appears incremental as it builds on existing theoretical developments.
The paper tackles the problem of comparing decision options under severe uncertainty with multiple targets and mixed measurement scales by transferring abstract decision theory to this setting, resulting in more informative orders than Pareto that can be computed via linear optimization.
The quality of consequences in a decision making problem under (severe) uncertainty must often be compared among different targets (goals, objectives) simultaneously. In addition, the evaluations of a consequence's performance under the various targets often differ in their scale of measurement, classically being either purely ordinal or perfectly cardinal. In this paper, we transfer recent developments from abstract decision theory with incomplete preferential and probabilistic information to this multi-target setting and show how -- by exploiting the (potentially) partial cardinal and partial probabilistic information -- more informative orders for comparing decisions can be given than the Pareto order. We discuss some interesting properties of the proposed orders between decision options and show how they can be concretely computed by linear optimization. We conclude the paper by demonstrating our framework in an artificial (but quite real-world) example in the context of comparing algorithms under different performance measures.