Tournament selection in zeroth-level classifier systems based on average reward reinforcement learning
This is an incremental improvement for researchers in genetics-based machine learning, enabling ZCS to solve larger multi-step problems.
The paper tackled the limitation of zeroth-level classifier systems (ZCS) in handling large multi-step problems by replacing its discounted reward reinforcement learning with R-learning and using tournament selection instead of roulette wheel selection, resulting in systems capable of supporting long action chains.
As a genetics-based machine learning technique, zeroth-level classifier system (ZCS) is based on a discounted reward reinforcement learning algorithm, bucket-brigade algorithm, which optimizes the discounted total reward received by an agent but is not suitable for all multi-step problems, especially large-size ones. There are some undiscounted reinforcement learning methods available, such as R-learning, which optimize the average reward per time step. In this paper, R-learning is used as the reinforcement learning employed by ZCS, to replace its discounted reward reinforcement learning approach, and tournament selection is used to replace roulette wheel selection in ZCS. The modification results in classifier systems that can support long action chains, and thus is able to solve large multi-step problems.