Accelerating Deep Reinforcement Learning for Digital Twin Network Optimization with Evolutionary Strategies
This work addresses scalability issues for network operators managing complex communication networks, but it is incremental as it applies a known method (ES) to a specific bottleneck in DRL.
The paper tackled the poor scalability of Deep Reinforcement Learning (DRL) in Digital Twin Network optimization by using Evolutionary Strategies (ES) for training, achieving speed-ups of 128x and 6x in training time for NSFNET and GEANT2 topologies.
The recent growth of emergent network applications (e.g., satellite networks, vehicular networks) is increasing the complexity of managing modern communication networks. As a result, the community proposed the Digital Twin Networks (DTN) as a key enabler of efficient network management. Network operators can leverage the DTN to perform different optimization tasks (e.g., Traffic Engineering, Network Planning). Deep Reinforcement Learning (DRL) showed a high performance when applied to solve network optimization problems. In the context of DTN, DRL can be leveraged to solve optimization problems without directly impacting the real-world network behavior. However, DRL scales poorly with the problem size and complexity. In this paper, we explore the use of Evolutionary Strategies (ES) to train DRL agents for solving a routing optimization problem. The experimental results show that ES achieved a training time speed-up of 128 and 6 for the NSFNET and GEANT2 topologies respectively.