Edge-Compatible Reinforcement Learning for Recommendations
This work addresses the challenge of deploying RL recommendation systems in asynchronous edge environments, offering a solution for real-time, distributed applications, though it appears incremental as it builds on existing coagent algorithms.
The authors tackled the problem of reinforcement learning (RL) recommendation systems in edge computing, which often rely on synchronization or unprincipled methods, by proposing a principled, asynchronous algorithm based on coagent policy gradients that functions effectively even with network degradation.
Most reinforcement learning (RL) recommendation systems designed for edge computing must either synchronize during recommendation selection or depend on an unprincipled patchwork collection of algorithms. In this work, we build on asynchronous coagent policy gradient algorithms \citep{kostas2020asynchronous} to propose a principled solution to this problem. The class of algorithms that we propose can be distributed over the internet and run asynchronously and in real-time. When a given edge fails to respond to a request for data with sufficient speed, this is not a problem; the algorithm is designed to function and learn in the edge setting, and network issues are part of this setting. The result is a principled, theoretically grounded RL algorithm designed to be distributed in and learn in this asynchronous environment. In this work, we describe this algorithm and a proposed class of architectures in detail, and demonstrate that they work well in practice in the asynchronous setting, even as the network quality degrades.