SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems
This work addresses efficiency issues for RL practitioners by accelerating training on real hardware, though it appears incremental as it applies existing RL algorithms to new PIM systems.
The paper tackles memory limitations in reinforcement learning training by exploring Processing-In-Memory architectures, achieving near-linear performance scaling and superior results compared to CPU and GPU implementations on OpenAI GYM environments.
Reinforcement Learning (RL) trains agents to learn optimal behavior by maximizing reward signals from experience datasets. However, RL training often faces memory limitations, leading to execution latencies and prolonged training times. To overcome this, SwiftRL explores Processing-In-Memory (PIM) architectures to accelerate RL workloads. We achieve near-linear performance scaling by implementing RL algorithms like Tabular Q-learning and SARSA on UPMEM PIM systems and optimizing for hardware. Our experiments on OpenAI GYM environments using UPMEM hardware demonstrate superior performance compared to CPU and GPU implementations.