Throughput and Latency in the Distributed Q-Learning Random Access mMTC Networks
This work addresses random access collisions for devices in mMTC networks, representing an incremental improvement over existing Q-learning methods.
The paper tackles the random access problem in massive machine-type communication networks by proposing a distributed packet-based Q-learning method that adjusts rewards to favor devices with more remaining packets, achieving a better throughput-latency trade-off than independent and collaborative techniques in practical scenarios.
In mMTC mode, with thousands of devices trying to access network resources sporadically, the problem of random access (RA) and collisions between devices that select the same resources becomes crucial. A promising approach to solve such an RA problem is to use learning mechanisms, especially the Q-learning algorithm, where the devices learn about the best time-slot periods to transmit through rewards sent by the central node. In this work, we propose a distributed packet-based learning method by varying the reward from the central node that favors devices having a larger number of remaining packets to transmit. Our numerical results indicated that the proposed distributed packet-based Q-learning method attains a much better throughput-latency trade-off than the alternative independent and collaborative techniques in practical scenarios of interest. In contrast, the number of payload bits of the packet-based technique is reduced regarding the collaborative Q-learning RA technique for achieving the same normalized throughput.