Random Actions vs Random Policies: Bootstrapping Model-Based Direct Policy Search
This work addresses the efficiency of bootstrapping dynamics models in reinforcement learning, but it is incremental as it compares existing methods without introducing new paradigms.
The paper investigates how different initial data gathering methods affect the learning of dynamics models for policy search, finding that task-dependent factors can harm each method and suggesting hybrid approaches.
This paper studies the impact of the initial data gathering method on the subsequent learning of a dynamics model. Dynamics models approximate the true transition function of a given task, in order to perform policy search directly on the model rather than on the costly real system. This study aims to determine how to bootstrap a model as efficiently as possible, by comparing initialization methods employed in two different policy search frameworks in the literature. The study focuses on the model performance under the episode-based framework of Evolutionary methods using probabilistic ensembles. Experimental results show that various task-dependant factors can be detrimental to each method, suggesting to explore hybrid approaches.