A double competitive strategy based learning automata algorithm
This work addresses a specific bottleneck in reinforcement learning for researchers, though it appears incremental as it builds on existing estimator algorithms.
The authors tackled the slow convergence of learning automata estimator algorithms by proposing a double competitive strategy that corrects wrong rewards instantly, achieving faster convergence than classic and state-of-the-art methods in benchmark tests.
Learning Automata (LA) are considered as one of the most powerful tools in the field of reinforcement learning. The family of estimator algorithms is proposed to improve the convergence rate of LA and has made great achievements. However, the estimators perform poorly on estimating the reward probabilities of actions in the initial stage of the learning process of LA. In this situation, a lot of rewards would be added to the probabilities of non-optimal actions. Thus, a large number of extra iterations are needed to compensate for these wrong rewards. In order to improve the speed of convergence, we propose a new P-model absorbing learning automaton by utilizing a double competitive strategy which is designed for updating the action probability vector. In this way, the wrong rewards can be corrected instantly. Hence, the proposed Double Competitive Algorithm overcomes the drawbacks of existing estimator algorithms. A refined analysis is presented to show the $ε-optimality$ of the proposed scheme. The extensive experimental results in benchmark environments demonstrate that our proposed learning automata perform more efficiently than the most classic LA $SE_{RI}$ and the current fastest LA $DGCPA^{*}$.