Multi-Agent Deep Reinforcement Learning for Distributed Satellite Routing
This is an incremental improvement for satellite network routing, building on previous Q-routing work.
This paper tackles the problem of routing in Low Earth Orbit Satellite Constellations by introducing a Multi-Agent Deep Reinforcement Learning approach, where each satellite acts as an independent agent with partial knowledge, and results show it efficiently learns optimal routes offline for distributed routing online.
This paper introduces a Multi-Agent Deep Reinforcement Learning (MA-DRL) approach for routing in Low Earth Orbit Satellite Constellations (LSatCs). Each satellite is an independent decision-making agent with a partial knowledge of the environment, and supported by feedback received from the nearby agents. Building on our previous work that introduced a Q-routing solution, the contribution of this paper is to extend it to a deep learning framework able to quickly adapt to the network and traffic changes, and based on two phases: (1) An offline exploration learning phase that relies on a global Deep Neural Network (DNN) to learn the optimal paths at each possible position and congestion level; (2) An online exploitation phase with local, on-board, pre-trained DNNs. Results show that MA-DRL efficiently learns optimal routes offline that are then loaded for an efficient distributed routing online.