Marina Petrova

Milad Ganjalizadeh, Hossein Shokri Ghadikolaei, Deniz Gündüz et al.

Recent advances in distributed intelligence have driven impressive progress across a diverse range of applications, from industrial automation to autonomous transportation. Nevertheless, deploying distributed learning services over wireless networks poses numerous challenges. These arise from inherent uncertainties in wireless environments (e.g., random channel fluctuations), limited resources (e.g., bandwidth and transmit power), and the presence of coexisting services on the network. In this paper, we investigate a mixed service scenario wherein high-priority ultra-reliable low latency communication (URLLC) and low-priority distributed learning services run concurrently over a network. Utilizing device selection, we aim to minimize the convergence time of distributed learning while simultaneously fulfilling the requirements of the URLLC service. We formulate this problem as a Markov decision process and address it via BSAC-CoEx, a framework based on the branching soft actor-critic (BSAC) algorithm that determines each device's participation decision through distinct branches in the actor's neural network. We evaluate our solution with a realistic simulator that is compliant with 3GPP standards for factory automation use cases. Our simulation results confirm that our solution can significantly decrease the training delays of the distributed learning service while keeping the URLLC availability above its required threshold and close to the scenario where URLLC solely consumes all wireless resources.

Wei Shi, Milad Ganjalizadeh, Hossein Shokri Ghadikolaei et al.

Ultra-reliable low latency communications (URLLC) service is envisioned to enable use cases with strict reliability and latency requirements in 5G. One approach for enabling URLLC services is to leverage Reinforcement Learning (RL) to efficiently allocate wireless resources. However, with conventional RL methods, the decision variables (though being deployed at various network layers) are typically optimized in the same control loop, leading to significant practical limitations on the control loop's delay as well as excessive signaling and energy consumption. In this paper, we propose a multi-agent Hierarchical RL (HRL) framework that enables the implementation of multi-level policies with different control loop timescales. Agents with faster control loops are deployed closer to the base station, while the ones with slower control loops are at the edge or closer to the core network providing high-level guidelines for low-level actions. On a use case from the prior art, with our HRL framework, we optimized the maximum number of retransmissions and transmission power of industrial devices. Our extensive simulation results on the factory automation scenario show that the HRL framework achieves better performance as the baseline single-agent RL method, with significantly less overhead of signal transmissions and delay compared to the one-agent RL methods.

S. Senthilmurugan, Junaid Ansari, Petri Mähönen et al.

We consider a multichannel Cognitive Radio Network (CRN), where secondary users sequentially sense channels for opportunistic spectrum access. In this scenario, the Channel Selection Algorithm (CSA) allows secondary users to find a vacant channel with the minimal number of channel switches. Most of the existing CSA literature assumes exponential ON-OFF time distribution for primary users (PU) channel occupancy pattern. This exponential assumption might be helpful to get performance bounds; but not useful to evaluate the performance of CSA under realistic conditions. An in-depth analysis of independent spectrum measurement traces reveals that wireless channels have typically heavy-tailed PU OFF times. In this paper, we propose an extension to the Predictive CSA framework and its generalization for heavy tailed PU OFF time distribution, which represents realistic scenarios. In particular, we calculate the probability of channel being idle for hyper-exponential OFF times to use in CSA. We implement our proposed CSA framework in a wireless test-bed and comprehensively evaluate its performance by recreating the realistic PU channel occupancy patterns. The proposed CSA shows significant reduction in channel switches and energy consumption as compared to Predictive CSA which always assumes exponential PU ON-OFF times.Through our work, we show the impact of the PU channel occupancy pattern on the performance of CSA in multichannel CRN.