Toward Real-Time Decentralized Reinforcement Learning using Finite Support Basis Functions
This addresses the problem of real-time robotic control in soccer robotics, offering an incremental improvement for efficient deployment on limited hardware.
The paper tackles the challenge of implementing real-time reinforcement learning on resource-constrained robots by proposing a decentralized approach with finite support basis functions, achieving up to 99.94% faster execution and 98.82% reduced memory usage without performance loss.
This paper addresses the design and implementation of complex Reinforcement Learning (RL) behaviors where multi-dimensional action spaces are involved, as well as the need to execute the behaviors in real-time using robotic platforms with limited computational resources and training times. For this purpose, we propose the use of decentralized RL, in combination with finite support basis functions as alternatives to Gaussian RBF, in order to alleviate the effects of the curse of dimensionality on the action and state spaces respectively, and to reduce the computation time. As testbed, a RL based controller for the in-walk kick in NAO robots, a challenging and critical problem for soccer robotics, is used. The reported experiments show empirically that our solution saves up to 99.94% of execution time and 98.82% of memory consumption during execution, without diminishing performance compared to classical approaches.