Benchmarking Deep Reinforcement Learning Algorithms for Vision-based Robotics
It provides a novel benchmarking study for vision-based robotics problems, addressing a gap in the field, but is incremental as it applies existing methods to new data.
This paper benchmarks state-of-the-art reinforcement learning algorithms, including SAC, PPO, and IPG with HER variants, on two simulated vision-based robotics problems (KukaDiverseObjectEnv and RacecarZEDGymEnv) using RGB images and continuous actions, establishing improvements through strategies like intermediate hindsight goals and feature extraction architectures.
This paper presents a benchmarking study of some of the state-of-the-art reinforcement learning algorithms used for solving two simulated vision-based robotics problems. The algorithms considered in this study include soft actor-critic (SAC), proximal policy optimization (PPO), interpolated policy gradients (IPG), and their variants with Hindsight Experience replay (HER). The performances of these algorithms are compared against PyBullet's two simulation environments known as KukaDiverseObjectEnv and RacecarZEDGymEnv respectively. The state observations in these environments are available in the form of RGB images and the action space is continuous, making them difficult to solve. A number of strategies are suggested to provide intermediate hindsight goals required for implementing HER algorithm on these problems which are essentially single-goal environments. In addition, a number of feature extraction architectures are proposed to incorporate spatial and temporal attention in the learning process. Through rigorous simulation experiments, the improvement achieved with these components are established. To the best of our knowledge, such a benchmarking study is not available for the above two vision-based robotics problems making it a novel contribution in the field.