rQdia: Regularizing Q-Value Distributions With Image Augmentation
This addresses the challenge of sample inefficiency in deep reinforcement learning for continuous control and Atari games, offering a simple method to boost performance, though it appears incremental as it builds on existing augmentation techniques.
The paper tackled the problem of improving sample efficiency and performance in pixel-based deep reinforcement learning by regularizing Q-value distributions with image augmentation, achieving gains on 9/12 MuJoCo tasks with DrQ and SAC and 18/26 Atari environments with Data-Efficient Rainbow, and enabling model-free continuous control from pixels to surpass state encoding baselines.
rQdia regularizes Q-value distributions with augmented images in pixel-based deep reinforcement learning. With a simple auxiliary loss, that equalizes these distributions via MSE, rQdia boosts DrQ and SAC on 9/12 and 10/12 tasks respectively in the MuJoCo Continuous Control Suite from pixels, and Data-Efficient Rainbow on 18/26 Atari Arcade environments. Gains are measured in both sample efficiency and longer-term training. Moreover, the addition of rQdia finally propels model-free continuous control from pixels over the state encoding baseline.