Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
This addresses exploration issues in deep reinforcement learning, but it is incremental as it builds on existing entropy-based methods.
The paper tackles the problem of exploration in deep reinforcement learning by showing that low initial entropy increases learning failure and biases exploration, and it introduces an entropy-aware model initialization strategy that reduces failures and improves performance, stability, and speed.
Encouraging exploration is a critical issue in deep reinforcement learning. We investigate the effect of initial entropy that significantly influences the exploration, especially at the earlier stage. Our main observations are as follows: 1) low initial entropy increases the probability of learning failure, and 2) this initial entropy is biased towards a low value that inhibits exploration. Inspired by the investigations, we devise entropy-aware model initialization, a simple yet powerful learning strategy for effective exploration. We show that the devised learning strategy significantly reduces learning failures and enhances performance, stability, and learning speed through experiments.