Model-based Policy Optimization using Symbolic World Model
This addresses sample efficiency challenges in robotics control, though it appears incremental as it builds on existing model-based reinforcement learning approaches.
The paper tackles the problem of low sample efficiency in model-free reinforcement learning for robotics by using symbolic regression to approximate environment dynamics, achieving superior sample efficiency compared to baseline methods in simulated tasks.
The application of learning-based control methods in robotics presents significant challenges. One is that model-free reinforcement learning algorithms use observation data with low sample efficiency. To address this challenge, a prevalent approach is model-based reinforcement learning, which involves employing an environment dynamics model. We suggest approximating transition dynamics with symbolic expressions, which are generated via symbolic regression. Approximation of a mechanical system with a symbolic model has fewer parameters than approximation with neural networks, which can potentially lead to higher accuracy and quality of extrapolation. We use a symbolic dynamics model to generate trajectories in model-based policy optimization to improve the sample efficiency of the learning algorithm. We evaluate our approach across various tasks within simulated environments. Our method demonstrates superior sample efficiency in these tasks compared to model-free and model-based baseline methods.