Distributed Proximal Policy Optimization for Contention-Based Spectrum Access
This work addresses spectrum access inefficiencies for wireless networks, but it is incremental as it applies an existing policy gradient method to a specific domain.
The paper tackles the problem of contention-based spectrum access in unlicensed wireless networks by developing a distributed Proximal Policy Optimization method, which significantly outperforms a genie-aided adaptive energy detection threshold in proportional fairness reward and improves sum and maximum user throughputs.
The increasing number of wireless devices operating in unlicensed spectrum motivates the development of intelligent adaptive approaches to spectrum access that go beyond traditional carrier sensing. We develop a novel distributed implementation of a policy gradient method known as Proximal Policy Optimization modelled on a two stage Markov decision process that enables such an intelligent approach, and still achieves decentralized contention-based medium access. In each time slot, a base station (BS) uses information from spectrum sensing and reception quality to autonomously decide whether or not to transmit on a given resource, with the goal of maximizing proportional fairness network-wide. Empirically, we find the proportional fairness reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold. This is further validated by the improved sum and maximum user throughputs achieved by our approach.