Adaptive Thompson Sampling Stacks for Memory Bounded Open-Loop Planning
This work addresses memory-bounded planning for partially observable environments, which is an incremental improvement over existing open-loop algorithms.
The paper tackles the problem of partially observable open-loop planning with memory constraints by proposing SYMBOL, a method that uses adaptive Thompson Sampling stacks, and demonstrates its effectiveness and robustness in large POMDP benchmarks with empirical evaluations.
We propose Stable Yet Memory Bounded Open-Loop (SYMBOL) planning, a general memory bounded approach to partially observable open-loop planning. SYMBOL maintains an adaptive stack of Thompson Sampling bandits, whose size is bounded by the planning horizon and can be automatically adapted according to the underlying domain without any prior domain knowledge beyond a generative model. We empirically test SYMBOL in four large POMDP benchmark problems to demonstrate its effectiveness and robustness w.r.t. the choice of hyperparameters and evaluate its adaptive memory consumption. We also compare its performance with other open-loop planning algorithms and POMCP.