Learn a Prior for RHEA for Better Online Planning
This work addresses performance bottlenecks in real-time game planning algorithms for AI researchers, though it is incremental as it builds on existing RHEA methods.
The paper tackled the performance limitations of Rolling Horizon Evolutionary Algorithms (RHEA) in online planning for real-time games by learning a prior offline with value and policy networks, resulting in significant performance improvements on OpenAI Gym MuJoCo tasks compared to standard RHEA.
Rolling Horizon Evolutionary Algorithms (RHEA) are a class of online planning methods for real-time game playing; their performance is closely related to the planning horizon and the search time allowed. In this paper, we propose to learn a prior for RHEA in an offline manner by training a value network and a policy network. The value network is used to reduce the planning horizon by providing an estimation of future rewards, and the policy network is used to initialize the population, which helps to narrow down the search scope. The proposed algorithm, named prior-based RHEA (p-RHEA), trains policy and value networks by performing planning and learning iteratively. In the planning stage, the horizon-limited search assisted with the policy network and value network is performed to improve the policies and collect training samples. In the learning stage, the policy network and value network are trained with the collected samples to learn better prior knowledge. Experimental results on OpenAI Gym MuJoCo tasks show that the performance of the proposed p-RHEA is significantly improved compared to that of RHEA.