Long-Term Mapping of the Douro River Plume with Multi-Agent Reinforcement Learning
This addresses the challenge of efficient, long-term environmental monitoring of dynamic river plumes for oceanographic applications, representing a domain-specific incremental advance.
The paper tackles the problem of long-term mapping of river plumes using multiple autonomous underwater vehicles (AUVs), proposing a multi-agent reinforcement learning approach that integrates spatiotemporal Gaussian process regression with a Q-network controller. Results show the method outperforms benchmarks, with doubling the number of AUVs more than doubling endurance while maintaining or improving accuracy, and policies generalize across unseen seasonal regimes.
We study the problem of long-term (multiple days) mapping of a river plume using multiple autonomous underwater vehicles (AUVs), focusing on the Douro river representative use-case. We propose an energy - and communication - efficient multi-agent reinforcement learning approach in which a central coordinator intermittently communicates with the AUVs, collecting measurements and issuing commands. Our approach integrates spatiotemporal Gaussian process regression (GPR) with a multi-head Q-network controller that regulates direction and speed for each AUV. Simulations using the Delft3D ocean model demonstrate that our method consistently outperforms both single- and multi-agent benchmarks, with scaling the number of agents both improving mean squared error (MSE) and operational endurance. In some instances, our algorithm demonstrates that doubling the number of AUVs can more than double endurance while maintaining or improving accuracy, underscoring the benefits of multi-agent coordination. Our learned policies generalize across unseen seasonal regimes over different months and years, demonstrating promise for future developments of data-driven long-term monitoring of dynamic plume environments.