Robust Active Measuring under Model Uncertainty
This work addresses decision-making challenges in uncertain environments for AI agents, offering an incremental improvement by extending existing MDP frameworks with active measuring and robustness.
The paper tackles the problem of sequential decision-making under partial observability and model uncertainty by introducing robust active-measuring MDPs (RAM-MDPs), showing that model uncertainty can reduce measurement needs and proposing a method to control this with bounded cost, with empirical results demonstrating superior scalability and performance compared to baselines.
Partial observability and uncertainty are common problems in sequential decision-making that particularly impede the use of formal models such as Markov decision processes (MDPs). However, in practice, agents may be able to employ costly sensors to measure their environment and resolve partial observability by gathering information. Moreover, imprecise transition functions can capture model uncertainty. We combine these concepts and extend MDPs to robust active-measuring MDPs (RAM-MDPs). We present an active-measure heuristic to solve RAM-MDPs efficiently and show that model uncertainty can, counterintuitively, let agents take fewer measurements. We propose a method to counteract this behavior while only incurring a bounded additional cost. We empirically compare our methods to several baselines and show their superior scalability and performance.