One is More: Diverse Perspectives within a Single Network for Efficient DRL
This work addresses efficiency issues in deep reinforcement learning for practitioners, though it appears incremental as it builds on existing algorithms with minimal overhead.
The paper tackles the challenges of low sample efficiency and overfitting in deep reinforcement learning by introducing OMNet, a novel paradigm that uses multiple subnetworks within a single network to provide diverse outputs efficiently, achieving an effective balance between performance and computational cost on the MuJoCo benchmark.
Deep reinforcement learning has achieved remarkable performance in various domains by leveraging deep neural networks for approximating value functions and policies. However, using neural networks to approximate value functions or policy functions still faces challenges, including low sample efficiency and overfitting. In this paper, we introduce OMNet, a novel learning paradigm utilizing multiple subnetworks within a single network, offering diverse outputs efficiently. We provide a systematic pipeline, including initialization, training, and sampling with OMNet. OMNet can be easily applied to various deep reinforcement learning algorithms with minimal additional overhead. Through comprehensive evaluations conducted on MuJoCo benchmark, our findings highlight OMNet's ability to strike an effective balance between performance and computational cost.