Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs
This work addresses channel access efficiency in WLANs, offering a practical solution for network performance enhancement, though it is incremental as it builds on existing MARL methods.
The paper tackles distributed channel access in wireless networks by proposing a heterogeneous multi-agent reinforcement learning framework called QPMIX, which improves throughput, reduces delay and collisions, and ensures fairness compared to conventional CSMA/CA mechanisms.
This paper investigates the use of multi-agent reinforcement learning (MARL) to address distributed channel access in wireless local area networks. In particular, we consider the challenging yet more practical case where the agents heterogeneously adopt value-based or policy-based reinforcement learning algorithms to train the model. We propose a heterogeneous MARL training framework, named QPMIX, which adopts a centralized training with distributed execution paradigm to enable heterogeneous agents to collaborate. Moreover, we theoretically prove the convergence of the proposed heterogeneous MARL method when using the linear value function approximation. Our method maximizes the network throughput and ensures fairness among stations, therefore, enhancing the overall network performance. Simulation results demonstrate that the proposed QPMIX algorithm improves throughput, mean delay, delay jitter, and collision rates compared with conventional carrier-sense multiple access with collision avoidance (CSMA/CA) mechanism in the saturated traffic scenario. Furthermore, the QPMIX algorithm is robust in unsaturated and delay-sensitive traffic scenarios. It coexists well with the conventional CSMA/CA mechanism and promotes cooperation among heterogeneous agents.