Temporally Encoded Double DQN for Proactive PRB Allocation in O-RAN Enabled Industrial Networks
This work aims to improve resource allocation and QoS for industrial networks, which are crucial for smart manufacturing applications, by addressing the limitations of existing static/reactive schedulers.
This paper addresses the challenge of sub-optimal resource utilization and QoS violations in 5G industrial networks due to static or reactive schedulers. It proposes a temporal-aware deep reinforcement learning (DRL) xApp that integrates an LSTM encoder within a Double DQN to proactively allocate Physical Resource Blocks (PRBs), improving slice satisfaction and buffer stability under moderate and heavy loads.
Fifth-generation (5G) wireless systems are increasingly adopted in smart manufacturing to support heterogeneous industrial workloads through services such as enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low-Latency Communication (URLLC). However, industrial traffic is inherently process-driven and temporally correlated. So, static or reactive schedulers in the Open Radio Access Network (O-RAN) are inadequate for such non-stationary conditions, leading to sub-optimal utilization and violation of latency-reliability guarantees. This paper proposes a temporal-aware deep reinforcement learning (DRL) xApp for proactive Physical Resource Block (PRB) allocation in O-RAN-enabled industrial networks. The proposed framework integrates a long short-term memory (LSTM) encoder within a Double Deep Q-Network (DQN) to model sequential dependencies among slice-level Key Performance Indicators (KPIs), enabling predictive and stable decision-making. A continuous-time Markov chain (CTMC) traffic model is incorporated to emulate machine concurrency and process burstiness. Experimental results show that the LSTM-Double DQN improves slice satisfaction, and buffer stability under moderate and heavy load, with the longest sequence window providing the strongest gains.