Cooperative Target Detection with AUVs: A Dual-Timescale Hierarchical MARDL Approach
This work addresses a domain-specific problem for underwater reconnaissance missions, offering an incremental improvement in multi-agent reinforcement learning for covert AUV operations.
The paper tackles the challenge of enabling autonomous underwater vehicles (AUVs) to cooperate efficiently while avoiding detection in adversarial environments, proposing a dual-timescale hierarchical MARDL approach that achieves rapid convergence and outperforms benchmarks in maximizing long-term cooperative efficiency with covert operations.
Autonomous Underwater Vehicles (AUVs) have shown great potential for cooperative detection and reconnaissance. However, collaborative AUV communications introduce risks of exposure. In adversarial environments, achieving efficient collaboration while ensuring covert operations becomes a key challenge for underwater cooperative missions. In this paper, we propose a novel dual time-scale Hierarchical Multi-Agent Proximal Policy Optimization (H-MAPPO) framework. The high-level component determines the individuals participating in the task based on a central AUV, while the low-level component reduces exposure probabilities through power and trajectory control by the participating AUVs. Simulation results show that the proposed framework achieves rapid convergence, outperforms benchmark algorithms in terms of performance, and maximizes long-term cooperative efficiency while ensuring covert operations.