Gaussian Process Aggregation for Root-Parallel Monte Carlo Tree Search with Continuous Actions
This work addresses an underexplored issue in online planning for continuous action spaces, offering a practical improvement for applications where wall clock time is limited but high performance is needed.
The paper tackled the problem of aggregating statistics from different threads in root-parallel Monte Carlo Tree Search for continuous action spaces by using Gaussian Process Regression to estimate values for untried actions, and demonstrated that this method outperforms existing strategies across six domains with a modest increase in inference time.
Monte Carlo Tree Search is a cornerstone algorithm for online planning, and its root-parallel variant is widely used when wall clock time is limited but best performance is desired. In environments with continuous action spaces, how to best aggregate statistics from different threads is an important yet underexplored question. In this work, we introduce a method that uses Gaussian Process Regression to obtain value estimates for promising actions that were not trialed in the environment. We perform a systematic evaluation across 6 different domains, demonstrating that our approach outperforms existing aggregation strategies while requiring a modest increase in inference time.