Abdullah Alghafis

Rusheng Zhang, Romain Leteurtre, Benjamin Striner et al.

Partially Detected Intelligent Traffic Signal Control (PD-ITSC) systems that can optimize traffic signals based on limited detected information could be a cost-efficient solution for mitigating traffic congestion in the future. In this paper, we focus on a particular problem in PD-ITSC - adaptation to changing environments. To this end, we investigate different reinforcement learning algorithms, including Q-learning, Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Actor-Critic with Kronecker-Factored Trust Region (ACKTR). Our findings suggest that RL algorithms can find optimal strategies under partial vehicle detection; however, policy-based algorithms can adapt to changing environments more efficiently than value-based algorithms. We use these findings to draw conclusions about the value of different models for PD-ITSC systems.

Daoud Burghal, Rui Wang, Abdullah Alghafis et al.

Many wireless networks, including 5G NR (New Radio) and future beyond 5G cellular systems, are expected to operate on multiple frequency bands. This paper considers the band assignment (BA) problem in dual-band systems, where the basestation (BS) chooses one of the two available frequency bands (centimeter-wave and millimeter-wave bands) to communicate with the user equipment (UE). While the millimeter-wave band might offer higher data rate, there is a significant probability of outage during which the communication should be carried on the (more reliable) centimeter-wave band. With mobility, the BA can be perceived as a sequential problem, where the BS uses previously observed information to predict the best band for a future time step. We formulate the BA as a binary classification problem and propose supervised Machine Learning (ML) solutions. We study the problem when both the BS and the UE use (i) omnidirectional antennas and (ii) both use directional antennas. In the omnidirectional case, we derive analytical benchmark solutions based on the Gaussian Process (GP) assumption for the inter-band shadow fading. In the directional case, where the labeling is shown to be complex, we propose an efficient labeling approach based on the Viterbi Algorithm (VA). We compare the performances for two channel models: (i) a stochastic channel and (ii) a ray-tracing based channel.