On the Interplay Between Sparsity and Training in Deep Reinforcement Learning
This work addresses the problem of optimizing neural network architectures for reinforcement learning practitioners, but it is incremental as it builds on existing sparse and dense methods without introducing a new paradigm.
The paper investigated how different sparse architectures affect learning performance in deep reinforcement learning for image-based domains, finding that sparse structure significantly impacts performance and the optimal architecture depends on whether hidden layer weights are fixed or learned.
We study the benefits of different sparse architectures for deep reinforcement learning. In particular, we focus on image-based domains where spatially-biased and fully-connected architectures are common. Using these and several other architectures of equal capacity, we show that sparse structure has a significant effect on learning performance. We also observe that choosing the best sparse architecture for a given domain depends on whether the hidden layer weights are fixed or learned.