A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQ
This addresses the challenge of efficient real-time data delivery in multi-user networks, such as IoT or wireless systems, but is incremental as it builds on existing AoI and RL frameworks.
The paper tackles the problem of scheduling time-sensitive information transmissions to multiple users over error-prone channels to minimize the long-term average age of information (AoI), with a resource constraint limiting average transmissions. It introduces a reinforcement learning approach that achieves near-optimal policies without prior channel knowledge, showing through simulations that methods like DQN can reduce AoI by up to 30% compared to baseline policies.
Scheduling the transmission of time-sensitive information from a source node to multiple users over error-prone communication channels is studied with the goal of minimizing the long-term average age of information (AoI) at the users. A long-term average resource constraint is imposed on the source, which limits the average number of transmissions. The source can transmit only to a single user at each time slot, and after each transmission, it receives an instantaneous ACK/NACK feedback from the intended receiver, and decides when and to which user to transmit the next update. Assuming the channel statistics are known, the optimal scheduling policy is studied for both the standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols. Then, a reinforcement learning(RL) approach is introduced to find a near-optimal policy, which does not assume any a priori information on the random processes governing the channel states. Different RL methods including average-cost SARSAwith linear function approximation (LFA), upper confidence reinforcement learning (UCRL2), and deep Q-network (DQN) are applied and compared through numerical simulations