Geiringer Theorems: From Population Genetics to Computational Intelligence, Memory Evolutive Systems and Hebbian Learning
This work addresses a specific problem in computational intelligence for researchers and practitioners in reinforcement learning, offering incremental advancements by connecting existing theorems to new algorithmic applications.
The paper tackles the challenge of action evaluation at chance nodes in reinforcement learning and Monte-Carlo tree search by adapting the classical Geiringer theorem from population genetics, resulting in novel dynamic parallel algorithms that involve independent agents traversing directed graphs with loops.
The classical Geiringer theorem addresses the limiting frequency of occurrence of various alleles after repeated application of crossover. It has been adopted to the setting of evolutionary algorithms and, a lot more recently, reinforcement learning and Monte-Carlo tree search methodology to cope with a rather challenging question of action evaluation at the chance nodes. The theorem motivates novel dynamic parallel algorithms that are explicitly described in the current paper for the first time. The algorithms involve independent agents traversing a dynamically constructed directed graph that possibly has loops. A rather elegant and profound category-theoretic model of cognition in biological neural networks developed by a well-known French mathematician, professor Andree Ehresmann jointly with a neurosurgeon, Jan Paul Vanbremeersch over the last thirty years provides a hint at the connection between such algorithms and Hebbian learning.