Deep Reinforcement Learning for Uplink Scheduling in NOMA-URLLC Networks
This addresses the stringent reliability and latency requirements for IoT applications in wireless networks, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles the problem of Ultra Reliable Low Latency Communications (URLLC) scheduling in Non-Orthogonal Multiple Access (NOMA) uplink networks by proposing a novel Deep Reinforcement Learning algorithm called NOMA-PPO, which outperforms traditional protocols and DRL benchmarks in 3GPP scenarios and demonstrates robustness across various channel and traffic conditions.
This article addresses the problem of Ultra Reliable Low Latency Communications (URLLC) in wireless networks, a framework with particularly stringent constraints imposed by many Internet of Things (IoT) applications from diverse sectors. We propose a novel Deep Reinforcement Learning (DRL) scheduling algorithm, named NOMA-PPO, to solve the Non-Orthogonal Multiple Access (NOMA) uplink URLLC scheduling problem involving strict deadlines. The challenge of addressing uplink URLLC requirements in NOMA systems is related to the combinatorial complexity of the action space due to the possibility to schedule multiple devices, and to the partial observability constraint that we impose to our algorithm in order to meet the IoT communication constraints and be scalable. Our approach involves 1) formulating the NOMA-URLLC problem as a Partially Observable Markov Decision Process (POMDP) and the introduction of an agent state, serving as a sufficient statistic of past observations and actions, enabling a transformation of the POMDP into a Markov Decision Process (MDP); 2) adapting the Proximal Policy Optimization (PPO) algorithm to handle the combinatorial action space; 3) incorporating prior knowledge into the learning agent with the introduction of a Bayesian policy. Numerical results reveal that not only does our approach outperform traditional multiple access protocols and DRL benchmarks on 3GPP scenarios, but also proves to be robust under various channel and traffic configurations, efficiently exploiting inherent time correlations.