Tristan Schuler

Tristan K. Schuler, Chinthan Prasad, Georgiy Kiselev et al.

Station-Keeping short-duration high-altitude balloons (HABs) in a region of interest is a challenging path-planning problem due to partially observable, complex, and dynamic wind flows. Deep reinforcement learning is a popular strategy for solving the station-keeping problem. A custom simulation environment was developed to train and evaluate Deep Q-Learning (DQN) for short-duration HAB agents in the simulation. To train the agents on realistic winds, synthetic wind forecasts were generated from aggregated historical radiosonde data to apply horizontal kinematics to simulated agents. The synthetic forecasts were closely correlated with ECWMF ERA5 Reanalysis forecasts, providing a realistic simulated wind field and seasonal and altitudinal variances between the wind models. DQN HAB agents were then trained and evaluated across different seasonal months. To highlight differences and trends in months with vastly different wind fields, a Forecast Score algorithm was introduced to independently classify forecasts based on wind diversity, and trends between station-keeping success and the Forecast Score were evaluated across all seasons.

Adam Haroon, Tristan Schuler

High Altitude Balloons (HABs) can leverage stratospheric wind layers for limited horizontal control, enabling applications in reconnaissance, environmental monitoring, and communications networks. Existing multi-agent HAB coordination approaches use deterministic methods like Voronoi partitioning and extremum seeking control for large global constellations, which perform poorly for smaller teams and localized missions. While single-agent HAB control using reinforcement learning has been demonstrated on HABs, coordinated multi-agent reinforcement learning (MARL) has not yet been investigated. This work presents the first systematic application of multi-agent reinforcement learning (MARL) to HAB coordination for distributed area coverage. We extend our previously developed reinforcement learning simulation environment (RLHAB) to support cooperative multi-agent learning, enabling multiple agents to operate simultaneously in realistic atmospheric conditions. We adapt QMIX for HAB area coverage coordination, leveraging Centralized Training with Decentralized Execution to address atmospheric vehicle coordination challenges. Our approach employs specialized observation spaces providing individual state, environmental context, and teammate data, with hierarchical rewards prioritizing coverage while encouraging spatial distribution. We demonstrate that QMIX achieves similar performance to the theoretically optimal geometric deterministic method for distributed area coverage, validating the MARL approach and providing a foundation for more complex autonomous multi-HAB missions where deterministic methods become intractable.

Ravi teja Nallapu, Yinan Xu, Abraham Marquez et al.

The recent observation of interstellar objects, 1I/Oumuamua and 2I/Borisov cross the solar system opened new opportunities for planetary science and planetary defense. As the first confirmed objects originating outside of the solar system, there are myriads of origin questions to explore and discuss, including where they came from, how did they get here and what are they composed of. Besides, there is a need to be cognizant especially if such interstellar objects pass by the Earth of potential dangers of impact. Specifically, in the case of Oumuamua, which was detected after its perihelion, passed by the Earth at around 0.2 AU, with an estimated excess speed of 60 km/s relative to the Earth. Without enough forewarning time, a collision with such high-speed objects can pose a catastrophic danger to all life Earth. Such challenges underscore the importance of detection and exploration systems to study these interstellar visitors. The detection system can include a spacecraft constellation with zenith-pointing telescope spacecraft. After an event is detected, a spacecraft swarm can be deployed from Earth to flyby past the visitor. The flyby can then be designed to perform a proximity operation of interest. This work aims to develop algorithms to design these swarm missions through the IDEAS (Integrated Design Engineering & Automation of Swarms) architecture. Specifically, we develop automated algorithms to design an Earth-based detection constellation and a spacecraft swarm that generates detailed surface maps of the visitor during the rendezvous, along with their heliocentric cruise trajectories.