Sample-efficient Cross-Entropy Method for Real-time Planning
This addresses the problem of real-time planning and control for robotics or autonomous systems, though it is incremental as it builds on an existing method.
The paper tackled the sampling inefficiency of the Cross-Entropy Method in model-based reinforcement learning for real-time planning, achieving a 2.7-22x reduction in samples and a 1.2-10x performance increase in high-dimensional control tasks.
Trajectory optimizers for model-based reinforcement learning, such as the Cross-Entropy Method (CEM), can yield compelling results even in high-dimensional control tasks and sparse-reward environments. However, their sampling inefficiency prevents them from being used for real-time planning and control. We propose an improved version of the CEM algorithm for fast planning, with novel additions including temporally-correlated actions and memory, requiring 2.7-22x less samples and yielding a performance increase of 1.2-10x in high-dimensional control problems.