RLgraph: Modular Computation Graphs for Deep Reinforcement Learning
This addresses the problem of algorithmic instability and complexity in reinforcement learning for researchers and practitioners, though it is incremental as it builds on existing modular and distributed computing concepts.
The paper tackles the challenges of implementing, executing, and testing reinforcement learning tasks by introducing RLgraph, a library that separates logical composition, backend graph definition, and distributed execution, resulting in robust, testable, and high-performance implementations across frameworks and backends.
Reinforcement learning (RL) tasks are challenging to implement, execute and test due to algorithmic instability, hyper-parameter sensitivity, and heterogeneous distributed communication patterns. We argue for the separation of logical component composition, backend graph definition, and distributed execution. To this end, we introduce RLgraph, a library for designing and executing reinforcement learning tasks in both static graph and define-by-run paradigms. The resulting implementations are robust, incrementally testable, and yield high performance across different deep learning frameworks and distributed backends.