Perception-Based Beliefs for POMDPs with Visual Observations
This addresses a bottleneck in planning under uncertainty for robotics or autonomous systems with visual inputs, though it is incremental as it builds on traditional solvers.
The paper tackles the intractability of POMDPs with high-dimensional visual observations by introducing the Perception-based Beliefs for POMDPs (PBP) framework, which uses an image classifier to map observations to state distributions and incorporates uncertainty quantification, resulting in outperformance over deep RL methods and improved robustness against visual corruption.
Partially observable Markov decision processes (POMDPs) are a principled planning model for sequential decision-making under uncertainty. Yet, real-world problems with high-dimensional observations, such as camera images, remain intractable for traditional belief- and filtering-based solvers. To tackle this problem, we introduce the Perception-based Beliefs for POMDPs framework (PBP), which complements such solvers with a perception model. This model takes the form of an image classifier which maps visual observations to probability distributions over states. PBP incorporates these distributions directly into belief updates, so the underlying solver does not need to reason explicitly over high-dimensional observation spaces. We show that the belief update of PBP coincides with the standard belief update if the image classifier is exact. Moreover, to handle classifier imprecision, we incorporate uncertainty quantification and introduce two methods to adjust the belief update accordingly. We implement PBP using two traditional POMDP solvers and empirically show that (1) it outperforms existing end-to-end deep RL methods and (2) uncertainty quantification improves robustness of PBP against visual corruption.