Federated Neuroevolution O-RAN: Enhancing the Robustness of Deep Reinforcement Learning xApps
This work addresses reliability concerns for RAN intelligent control in O-RAN architectures, representing an incremental improvement in DRL methods for network optimization.
The paper tackles the problem of local optima in deep reinforcement learning (DRL) xApps for O-RAN by introducing Federated O-RAN enabled Neuroevolution-enhanced DRL (F-ONRL), which improves robustness and balances computational load, as demonstrated on the Open AI Cellular platform.
The open radio access network (O-RAN) architecture introduces RAN intelligent controllers (RICs) to facilitate the management and optimization of the disaggregated RAN. Reinforcement learning (RL) and its advanced form, deep RL (DRL), are increasingly employed for designing intelligent controllers, or xApps, to be deployed in the near-real time (near-RT) RIC. These models often encounter local optima, which raise concerns about their reliability for RAN intelligent control. We therefore introduce Federated O-RAN enabled Neuroevolution (NE)-enhanced DRL (F-ONRL) that deploys an NE-based optimizer xApp in parallel to the RAN controller xApps. This NE-DRL xApp framework enables effective exploration and exploitation in the near-RT RIC without disrupting RAN operations. We implement the NE xApp along with a DRL xApp and deploy them on Open AI Cellular (OAIC) platform and present numerical results that demonstrate the improved robustness of xApps while effectively balancing the additional computational load.