Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control
This work addresses the problem of network congestion control for data center operators, but it is incremental as it focuses on prototyping and benchmarking rather than deploying RL in production.
The paper tackles the challenge of applying deep reinforcement learning to data center traffic control by developing Iroko, an emulator for evaluating RL algorithms against traditional methods, showing that RL can outperform TCP New Vegas on specific topologies.
Recent networking research has identified that data-driven congestion control (CC) can be more efficient than traditional CC in TCP. Deep reinforcement learning (RL), in particular, has the potential to learn optimal network policies. However, RL suffers from instability and over-fitting, deficiencies which so far render it unacceptable for use in datacenter networks. In this paper, we analyze the requirements for RL to succeed in the datacenter context. We present a new emulator, Iroko, which we developed to support different network topologies, congestion control algorithms, and deployment scenarios. Iroko interfaces with the OpenAI gym toolkit, which allows for fast and fair evaluation of different RL and traditional CC algorithms under the same conditions. We present initial benchmarks on three deep RL algorithms compared to TCP New Vegas and DCTCP. Our results show that these algorithms are able to learn a CC policy which exceeds the performance of TCP New Vegas on a dumbbell and fat-tree topology. We make our emulator open-source and publicly available: https://github.com/dcgym/iroko