Gym-TORAX: Open-source software for integrating RL with plasma control simulators
This tool facilitates RL research in plasma control for fusion energy applications, but it is incremental as it builds on existing simulation frameworks.
The authors developed Gym-TORAX, an open-source Python package that integrates reinforcement learning (RL) with plasma control simulators for tokamaks, enabling users to create RL environments to optimize plasma characteristics like performance and stability, with one environment already available for an ITER ramp-up scenario.
This paper presents Gym-TORAX, a Python package enabling the implementation of Reinforcement Learning (RL) environments for simulating plasma dynamics and control in tokamaks. Users define succinctly a set of control actions and observations, and a control objective from which Gym-TORAX creates a Gymnasium environment that wraps TORAX for simulating the plasma dynamics. The objective is formulated through rewards depending on the simulated state of the plasma and control action to optimize specific characteristics of the plasma, such as performance and stability. The resulting environment instance is then compatible with a wide range of RL algorithms and libraries and will facilitate RL research in plasma control. In its current version, one environment is readily available, based on a ramp-up scenario of the International Thermonuclear Experimental Reactor (ITER).