Lineage Evolution Reinforcement Learning
This work addresses the incremental enhancement of reinforcement learning methods for AI agents in gaming environments.
The paper tackles the problem of improving reinforcement learning algorithms by introducing a lineage evolution approach that incorporates genetic algorithm modules and a lineage factor to retain agent potential, resulting in performance improvements in some Atari 2600 games.
We propose a general agent population learning system, and on this basis, we propose lineage evolution reinforcement learning algorithm. Lineage evolution reinforcement learning is a kind of derivative algorithm which accords with the general agent population learning system. We take the agents in DQN and its related variants as the basic agents in the population, and add the selection, mutation and crossover modules in the genetic algorithm to the reinforcement learning algorithm. In the process of agent evolution, we refer to the characteristics of natural genetic behavior, add lineage factor to ensure the retention of potential performance of agent, and comprehensively consider the current performance and lineage value when evaluating the performance of agent. Without changing the parameters of the original reinforcement learning algorithm, lineage evolution reinforcement learning can optimize different reinforcement learning algorithms. Our experiments show that the idea of evolution with lineage improves the performance of original reinforcement learning algorithm in some games in Atari 2600.