Adaptive Contention Window Design using Deep Q-learning
This work provides a method for wireless network nodes to intelligently adapt their contention window parameters, improving network-level utility for network operators and users, which is an incremental improvement.
This paper addresses the problem of dynamically adapting the minimum contention window (MCW) in random-access wireless networks to maximize network utility. The proposed deep Q-learning agent, using local channel observations, achieves near-optimal performance and significantly outperforms existing learning and non-learning methods.
We study the problem of adaptive contention window (CW) design for random-access wireless networks. More precisely, our goal is to design an intelligent node that can dynamically adapt its minimum CW (MCW) parameter to maximize a network-level utility knowing neither the MCWs of other nodes nor how these change over time. To achieve this goal, we adopt a reinforcement learning (RL) framework where we circumvent the lack of system knowledge with local channel observations and we reward actions that lead to high utilities. To efficiently learn these preferred actions, we follow a deep Q-learning approach, where the Q-value function is parametrized using a multi-layer perception. In particular, we implement a rainbow agent, which incorporates several empirical improvements over the basic deep Q-network. Numerical experiments based on the NS3 simulator reveal that the proposed RL agent performs close to optimal and markedly improves upon existing learning and non-learning based alternatives.