RBED: Reward Based Epsilon Decay
This work addresses the challenge of efficient exploration-exploitation trade-offs for reinforcement learning agents, presenting an incremental improvement over existing decay methods.
The paper tackles the problem of balancing exploration and exploitation in reinforcement learning by proposing a reward-based epsilon decay method, which in tested environments produces more consistent results and on average performs better than standard exponential decay.
$\varepsilon$-greedy is a policy used to balance exploration and exploitation in many reinforcement learning setting. In cases where the agent uses some on-policy algorithm to learn optimal behaviour, it makes sense for the agent to explore more initially and eventually exploit more as it approaches the target behaviour. This shift from heavy exploration to heavy exploitation can be represented as decay in the $\varepsilon$ value, where $\varepsilon$ depicts the how much an agent is allowed to explore. This paper proposes a new approach to this $\varepsilon$ decay where the decay is based on feedback from the environment. This paper also compares and contrasts one such approach based on rewards and compares it against standard exponential decay. The new approach, in the environments tested, produces more consistent results that on average perform better.