Reinforcement Learning with Goal-Distance Gradient
This addresses the challenge of training agents in environments with sparse or no rewards, which is a common issue in reinforcement learning, though it appears incremental as it builds on existing methods by replacing rewards with distance metrics.
The paper tackles the problem of sparse or absent rewards in reinforcement learning by proposing a model-free method that uses the minimum number of transitions between states as a distance metric to replace environmental rewards, with experiments showing better performance on sparse reward and local optimal problems in complex environments compared to previous work.
Reinforcement learning usually uses the feedback rewards of environmental to train agents. But the rewards in the actual environment are sparse, and even some environments will not rewards. Most of the current methods are difficult to get good performance in sparse reward or non-reward environments. Although using shaped rewards is effective when solving sparse reward tasks, it is limited to specific problems and learning is also susceptible to local optima. We propose a model-free method that does not rely on environmental rewards to solve the problem of sparse rewards in the general environment. Our method use the minimum number of transitions between states as the distance to replace the rewards of environmental, and proposes a goal-distance gradient to achieve policy improvement. We also introduce a bridge point planning method based on the characteristics of our method to improve exploration efficiency, thereby solving more complex tasks. Experiments show that our method performs better on sparse reward and local optimal problems in complex environments than previous work.