Task Specific Sharpness Aware O-RAN Resource Management using Multi Agent Reinforcement Learning
This work addresses resource management challenges for next-generation O-RAN networks, representing an incremental improvement with a novel hybrid method.
The paper tackles the problem of robustness and generalizability in dynamic O-RAN resource management by enhancing the Soft Actor Critic algorithm with a Sharpness-Aware Minimization mechanism in a multi-agent RL framework, resulting in up to a 22% improvement in resource allocation efficiency and superior QoS satisfaction.
Next-generation networks utilize the Open Radio Access Network (O-RAN) architecture to enable dynamic resource management, facilitated by the RAN Intelligent Controller (RIC). While deep reinforcement learning (DRL) models show promise in optimizing network resources, they often struggle with robustness and generalizability in dynamic environments. This paper introduces a novel resource management approach that enhances the Soft Actor Critic (SAC) algorithm with Sharpness-Aware Minimization (SAM) in a distributed Multi-Agent RL (MARL) framework. Our method introduces an adaptive and selective SAM mechanism, where regularization is explicitly driven by temporal-difference (TD)-error variance, ensuring that only agents facing high environmental complexity are regularized. This targeted strategy reduces unnecessary overhead, improves training stability, and enhances generalization without sacrificing learning efficiency. We further incorporate a dynamic $ρ$ scheduling scheme to refine the exploration-exploitation trade-off across agents. Experimental results show our method significantly outperforms conventional DRL approaches, yielding up to a $22\%$ improvement in resource allocation efficiency and ensuring superior QoS satisfaction across diverse O-RAN slices.