Marina Petrova

Wei Shi, Milad Ganjalizadeh, Hossein Shokri Ghadikolaei et al.

Ultra-reliable low latency communications (URLLC) service is envisioned to enable use cases with strict reliability and latency requirements in 5G. One approach for enabling URLLC services is to leverage Reinforcement Learning (RL) to efficiently allocate wireless resources. However, with conventional RL methods, the decision variables (though being deployed at various network layers) are typically optimized in the same control loop, leading to significant practical limitations on the control loop's delay as well as excessive signaling and energy consumption. In this paper, we propose a multi-agent Hierarchical RL (HRL) framework that enables the implementation of multi-level policies with different control loop timescales. Agents with faster control loops are deployed closer to the base station, while the ones with slower control loops are at the edge or closer to the core network providing high-level guidelines for low-level actions. On a use case from the prior art, with our HRL framework, we optimized the maximum number of retransmissions and transmission power of industrial devices. Our extensive simulation results on the factory automation scenario show that the HRL framework achieves better performance as the baseline single-agent RL method, with significantly less overhead of signal transmissions and delay compared to the one-agent RL methods.

S. Senthilmurugan, Junaid Ansari, Petri Mähönen et al.

We consider a multichannel Cognitive Radio Network (CRN), where secondary users sequentially sense channels for opportunistic spectrum access. In this scenario, the Channel Selection Algorithm (CSA) allows secondary users to find a vacant channel with the minimal number of channel switches. Most of the existing CSA literature assumes exponential ON-OFF time distribution for primary users (PU) channel occupancy pattern. This exponential assumption might be helpful to get performance bounds; but not useful to evaluate the performance of CSA under realistic conditions. An in-depth analysis of independent spectrum measurement traces reveals that wireless channels have typically heavy-tailed PU OFF times. In this paper, we propose an extension to the Predictive CSA framework and its generalization for heavy tailed PU OFF time distribution, which represents realistic scenarios. In particular, we calculate the probability of channel being idle for hyper-exponential OFF times to use in CSA. We implement our proposed CSA framework in a wireless test-bed and comprehensively evaluate its performance by recreating the realistic PU channel occupancy patterns. The proposed CSA shows significant reduction in channel switches and energy consumption as compared to Predictive CSA which always assumes exponential PU ON-OFF times.Through our work, we show the impact of the PU channel occupancy pattern on the performance of CSA in multichannel CRN.