Iterated risk measures for risk-sensitive Markov decision processes with discounted cost
This addresses a foundational issue in risk-sensitive decision-making for fields like economics and AI, though it appears incremental as it modifies existing utility frameworks.
The paper identifies a limitation of discounted expected utility in representing rational risk preferences under discounted costs, showing it cannot capture certain rational preferences and leads to time inconsistency. It demonstrates that an iterated risk measure can represent these preferences and ensure time-consistent decision-making.
We demonstrate a limitation of discounted expected utility, a standard approach for representing the preference to risk when future cost is discounted. Specifically, we provide an example of the preference of a decision maker that appears to be rational but cannot be represented with any discounted expected utility. A straightforward modification to discounted expected utility leads to inconsistent decision making over time. We will show that an iterated risk measure can represent the preference that cannot be represented by any discounted expected utility and that the decisions based on the iterated risk measure are consistent over time.