D2RL: Deep Dense Architectures in Reinforcement Learning
This work addresses the problem of improving reinforcement learning performance for researchers and practitioners in robotics, though it is incremental as it adapts existing architectural ideas to a new domain.
The paper tackled the under-exploration of neural network architectures in reinforcement learning by applying deeper networks and dense connections from computer vision and generative modeling to simulated robotic tasks, finding significant benefits across manipulation and locomotion tasks for both proprioceptive and image-based observations.
While improvements in deep learning architectures have played a crucial role in improving the state of supervised and unsupervised learning in computer vision and natural language processing, neural network architecture choices for reinforcement learning remain relatively under-explored. We take inspiration from successful architectural choices in computer vision and generative modelling, and investigate the use of deeper networks and dense connections for reinforcement learning on a variety of simulated robotic learning benchmark environments. Our findings reveal that current methods benefit significantly from dense connections and deeper networks, across a suite of manipulation and locomotion tasks, for both proprioceptive and image-based observations. We hope that our results can serve as a strong baseline and further motivate future research into neural network architectures for reinforcement learning. The project website with code is at this link https://sites.google.com/view/d2rl/home.