Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring
For autonomous underwater vehicle control in marine ecosystem monitoring, this work addresses the need for reusable and generalizable policies under uncertain dynamics, though it is an incremental application of existing multi-task RL methods to a new domain.
The paper proposes a contextual multi-task reinforcement learning approach for autonomous underwater reef monitoring, enabling a single policy to handle multiple tasks (e.g., detecting oysters vs. corals) and showing improved sample efficiency, zero-shot generalization to unseen tasks, and robustness to varying water currents in simulated environments.
Although autonomous underwater vehicles promise the capability of marine ecosystem monitoring, their deployment is fundamentally limited by the difficulty of controlling vehicles under highly uncertain and non-stationary underwater dynamics. To address these challenges, we employ a data-driven reinforcement learning approach to compensate for unknown dynamics and task variations.Traditional single-task reinforcement learning has a tendency to overfit the training environment, thus, limit the long-term usefulness of the learnt policy. Hence, we propose to use a contextual multi-task reinforcement learning paradigm instead, allowing us to learn controllers that can be reused for various tasks, e.g., detecting oysters in one reef and detecting corals in another. We evaluate whether contextual multi-task reinforcement learning can efficiently learn robust and generalisable control policies for autonomous underwater reef monitoring. We train a single context-dependent policy that is able to solve multiple related monitoring tasks in a simulated reef environment in HoloOcean. In our experiments, we empirically evaluate the contextual policies regarding sample-efficiency, zero-shot generalisation to unseen tasks, and robustness to varying water currents. By utilising multi-task reinforcement learning, we aim to improve the training effectiveness, as well as the reusability of learnt policies to take a step towards more sustainable procedures in autonomous reef monitoring.