Learn for Variation: Variationally Guided AAV Trajectory Learning in Differentiable Environments
This addresses training instability in AAV trajectory planning for 6G IoT networks, but it is an incremental improvement over existing methods.
The paper tackled the problem of unstable and inefficient reinforcement learning for autonomous aerial vehicle trajectory planning by proposing a gradient-informed framework that replaces sparse rewards with dense policy gradients, resulting in improved mission completion time, average transmission rate, and training cost compared to baselines like DQN and DDPG.
Autonomous aerial vehicles (AAVs) empower sixth-generation (6G) Internet-of-Things (IoT) networks through mobility-driven data collection. However, conventional reward-driven reinforcement learning for AAV trajectory planning suffers from severe credit assignment issues and training instability, because sparse scalar rewards fail to capture the long-term and nonlinear effects of sequential movements. To address these challenges, this paper proposes Learn for Variation (L4V), a gradient-informed trajectory learning framework that replaces high-variance scalar reward signals with dense and analytically grounded policy gradients. Particularly, the coupled evolution of AAV kinematics, distance-dependent channel gains, and per-user data-collection progress is first unrolled into an end-to-end differentiable computational graph. Backpropagation through time then serves as a discrete adjoint solver, which propagates exact sensitivities from the cumulative mission objective to every control action and policy parameter. These structured gradients are used to train a deterministic neural policy with temporal smoothness regularization and gradient clipping. Extensive simulations demonstrate that L4V consistently outperforms representative baselines, including a genetic algorithm, DQN, A2C, and DDPG, in mission completion time, average transmission rate, and training cost