Seasonal Station-Keeping of Short Duration High Altitude Balloons using Deep Reinforcement Learning
This work addresses a domain-specific path-planning challenge for high-altitude balloon operations, representing an incremental application of existing methods to new data.
The paper tackled the problem of station-keeping for short-duration high-altitude balloons in dynamic wind conditions by developing a custom simulation environment and using Deep Q-Learning, achieving evaluation across different seasonal months with trends in success rates linked to a Forecast Score algorithm.
Station-Keeping short-duration high-altitude balloons (HABs) in a region of interest is a challenging path-planning problem due to partially observable, complex, and dynamic wind flows. Deep reinforcement learning is a popular strategy for solving the station-keeping problem. A custom simulation environment was developed to train and evaluate Deep Q-Learning (DQN) for short-duration HAB agents in the simulation. To train the agents on realistic winds, synthetic wind forecasts were generated from aggregated historical radiosonde data to apply horizontal kinematics to simulated agents. The synthetic forecasts were closely correlated with ECWMF ERA5 Reanalysis forecasts, providing a realistic simulated wind field and seasonal and altitudinal variances between the wind models. DQN HAB agents were then trained and evaluated across different seasonal months. To highlight differences and trends in months with vastly different wind fields, a Forecast Score algorithm was introduced to independently classify forecasts based on wind diversity, and trends between station-keeping success and the Forecast Score were evaluated across all seasons.