A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement
This addresses the optimization of resource allocation in virtualized networks for improved efficiency, though it appears incremental as it builds on existing DRL and heuristic techniques.
The paper tackles the network slice placement problem by introducing a hybrid approach combining Deep Reinforcement Learning (DRL) with a heuristic based on the Power of Two Choices, which accelerates learning and improves resource usage compared to state-of-the-art methods.
Network Slice placement with the problem of allocation of resources from a virtualized substrate network is an optimization problem which can be formulated as a multiobjective Integer Linear Programming (ILP) problem. However, to cope with the complexity of such a continuous task and seeking for optimality and automation, the use of Machine Learning (ML) techniques appear as a promising approach. We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization heuristic based on the Power of Two Choices principle. The DRL algorithm uses the so-called Asynchronous Advantage Actor Critic (A3C) algorithm for fast learning, and Graph Convolutional Networks (GCN) to automate feature extraction from the physical substrate network. The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches as the evaluation results evidence.