Temporally Consistent Graph Q-Networks for Intelligent Network Control
For network operators managing complex mobile networks, this algorithm provides a way to optimize antenna parameters under dynamic objectives, improving energy efficiency without sacrificing QoS.
The paper proposes a multi-agent reinforcement learning algorithm (TC-GQN) for intelligent network control that learns a task-independent, self-predicting representation of the entire network. In simulations, it outperforms graph-based baselines and rule-based controllers by improving hardware sleep time while maintaining quality of service, and enables rapid adaptation to changing objectives.
Mobile networks continue to grow in complexity and next generation networks are expected to support both increasing traffic loads and more diverse services. As network complexity rises, optimizing antenna parameters under dynamic or changing objectives becomes increasingly challenging. We propose a novel multi-agent reinforcement learning (MARL) algorithm for high-level control and orchestration of mobile networks. The Temporally Consistent Graph Q-Network (TC-GQN) algorithm learns a self-predicting representation of the whole network that is task-independent and aggregates information from all base-stations. A graph neural network is trained using a global reward function to assign coordinated local actions based on the learned encoding of the global network state. We evaluate the algorithm in a simulated environment to orchestrate an energy-saving feature across multiple sectors and multiple carriers under different quality of service (QoS) constraints. The proposed algorithm outperforms state-of-the-art graph-based baselines and a competitive rule-based controller by improving hardware sleep time while maintaining QoS. Moreover, the learned representation enables rapid adaptation to changing intents.