Combining Offline Models and Online Monte-Carlo Tree Search for Planning from Scratch
This addresses the problem of planning under model uncertainty for AI systems, offering a scalable solution where prior methods fail, though it builds incrementally on existing techniques like PSRs and Monte-Carlo tree search.
The paper tackles planning in stochastic, partially observable environments by combining offline Predictive State Representations (PSRs) with online Monte-Carlo tree search, enabling planning from scratch without prior knowledge. It demonstrates effectiveness through convergence proofs and scalability on the RockSample problem, outperforming state-of-the-art BA-POMDP approaches.
Planning in stochastic and partially observable environments is a central issue in artificial intelligence. One commonly used technique for solving such a problem is by constructing an accurate model firstly. Although some recent approaches have been proposed for learning optimal behaviour under model uncertainty, prior knowledge about the environment is still needed to guarantee the performance of the proposed algorithms. With the benefits of the Predictive State Representations~(PSRs) approach for state representation and model prediction, in this paper, we introduce an approach for planning from scratch, where an offline PSR model is firstly learned and then combined with online Monte-Carlo tree search for planning with model uncertainty. By comparing with the state-of-the-art approach of planning with model uncertainty, we demonstrated the effectiveness of the proposed approaches along with the proof of their convergence. The effectiveness and scalability of our proposed approach are also tested on the RockSample problem, which are infeasible for the state-of-the-art BA-POMDP based approaches.