Observation Adaptation via Annealed Importance Resampling for Partially Observable Markov Decision Processes
This addresses a key bottleneck in sequential decision-making under uncertainty for robotics and AI applications, though it is an incremental improvement over existing particle filter methods.
The paper tackles the problem of particle degeneracy and sample impoverishment in online POMDP solvers due to misalignment between state transition models and posterior belief distributions, especially with informative observations, by proposing an approach using bridge distributions via iterative Monte Carlo steps, resulting in significantly superior performance compared to state-of-the-art methods across multiple challenging domains.
Partially observable Markov decision processes (POMDPs) are a general mathematical model for sequential decision-making in stochastic environments under state uncertainty. POMDPs are often solved \textit{online}, which enables the algorithm to adapt to new information in real time. Online solvers typically use bootstrap particle filters based on importance resampling for updating the belief distribution. Since directly sampling from the ideal state distribution given the latest observation and previous state is infeasible, particle filters approximate the posterior belief distribution by propagating states and adjusting weights through prediction and resampling steps. However, in practice, the importance resampling technique often leads to particle degeneracy and sample impoverishment when the state transition model poorly aligns with the posterior belief distribution, especially when the received observation is highly informative. We propose an approach that constructs a sequence of bridge distributions between the state-transition and optimal distributions through iterative Monte Carlo steps, better accommodating noisy observations in online POMDP solvers. Our algorithm demonstrates significantly superior performance compared to state-of-the-art methods when evaluated across multiple challenging POMDP domains.