RMBench: Benchmarking Deep Reinforcement Learning for Robotic Manipulator Control
This provides a standardized evaluation framework for researchers in robotics and reinforcement learning, though it is incremental as it benchmarks existing methods on new tasks.
The authors introduced RMBench, the first benchmark for evaluating deep reinforcement learning algorithms on robotic manipulation tasks with high-dimensional continuous action and state spaces, finding that soft Actor-Critic outperformed most algorithms in average reward and stability, but no algorithm handled all tasks well.
Reinforcement learning is applied to solve actual complex tasks from high-dimensional, sensory inputs. The last decade has developed a long list of reinforcement learning algorithms. Recent progress benefits from deep learning for raw sensory signal representation. One question naturally arises: how well do they perform concerning different robotic manipulation tasks? Benchmarks use objective performance metrics to offer a scientific way to compare algorithms. In this paper, we present RMBench, the first benchmark for robotic manipulations, which have high-dimensional continuous action and state spaces. We implement and evaluate reinforcement learning algorithms that directly use observed pixels as inputs. We report their average performance and learning curves to show their performance and stability of training. Our study concludes that none of the studied algorithms can handle all tasks well, soft Actor-Critic outperforms most algorithms in average reward and stability, and an algorithm combined with data augmentation may facilitate learning policies. Our code is publicly available at https://github.com/xiangyanfei212/RMBench-2022, including all benchmark tasks and studied algorithms.