Exploiting generalisation symmetries in accuracy-based learning classifier systems: An initial study
This is an incremental improvement for reinforcement learning systems using learning classifier systems.
The paper tackles the problem of learning classifier systems hindering symmetrical generalizations across actions by introducing rules with multiple actions and maintaining separate accuracy and reward metrics for each. It shows that this approach exploits problem symmetries to improve performance without degrading it when symmetries are reduced.
Modern learning classifier systems typically exploit a niched genetic algorithm to facilitate rule discovery. When used for reinforcement learning, such rules represent generalisations over the state-action-reward space. Whilst encouraging maximal generality, the niching can potentially hinder the formation of generalisations in the state space which are symmetrical, or very similar, over different actions. This paper introduces the use of rules which contain multiple actions, maintaining accuracy and reward metrics for each action. It is shown that problem symmetries can be exploited, improving performance, whilst not degrading performance when symmetries are reduced.