Pratheek S. Upadhyaya

Pratheek S. Upadhyaya, Vijay K. Shah, Jeffrey H. Reed

As several new spectrum bands are opening up for shared use, a new paradigm of \textit{Diverse Band-aware Dynamic Spectrum Access} (d-DSA) has emerged. d-DSA equips a secondary device with software defined radios (SDRs) and utilize whitespaces (or idle channels) in \textit{multiple bands}, including but not limited to TV, LTE, Citizen Broadband Radio Service (CBRS), unlicensed ISM. In this paper, we propose a decentralized, online multi-agent reinforcement learning based cross-layer BAnd selection and Routing Design (BARD) for such d-DSA networks. BARD not only harnesses whitespaces in multiple spectrum bands, but also accounts for unique electro-magnetic characteristics of those bands to maximize the desired quality of service (QoS) requirements of heterogeneous message packets; while also ensuring no harmful interference to the primary users in the utilized band. Our extensive experiments demonstrate that BARD outperforms the baseline dDSAaR algorithm in terms of message delivery ratio, however, at a relatively higher network latency, for varying number of primary and secondary users. Furthermore, BARD greatly outperforms its single-band DSA variants in terms of both the metrics in all considered scenarios.

Ramkumar Raghu, Pratheek Upadhyaya, Mahadesh Panju et al.

We consider a multicast scheme recently proposed for a wireless downlink in [1]. It was shown earlier that power control can significantly improve its performance. However for this system, obtaining optimal power control is intractable because of a very large state space. Therefore in this paper we use deep reinforcement learning where we use function approximation of the Q-function via a deep neural network. We show that optimal power control can be learnt for reasonably large systems via this approach. The average power constraint is ensured via a Lagrange multiplier, which is also learnt. Finally, we demonstrate that a slight modification of the learning algorithm allows the optimal control to track the time varying system statistics.