A Note on Stability in Asynchronous Stochastic Approximation without Communication Delays
This work addresses stability issues in asynchronous algorithms for researchers in reinforcement learning and optimization, but it is incremental as it builds on existing methods.
The paper tackles the stability of asynchronous stochastic approximation algorithms without communication delays by extending Borkar and Meyn's method to more general noise conditions, and it derives convergence results applicable to average-reward reinforcement learning problems.
In this paper, we study asynchronous stochastic approximation algorithms without communication delays. Our main contribution is a stability proof for these algorithms that extends a method of Borkar and Meyn by accommodating more general noise conditions. We also derive convergence results from this stability result and discuss their application in important average-reward reinforcement learning problems.