Enabling A Network AI Gym for Autonomous Cyber Agents
It addresses the problem of slow training for network cyber operations agents, offering a sim-to-real solution that is incremental in applying RL methods to this domain.
This work tackled the challenge of training autonomous cyber agents by developing CyGIL, a unified environment that uses emulation to generate simulation for faster training, enabling agents trained in simulation to transfer effectively to emulated networks with full proficiency.
This work aims to enable autonomous agents for network cyber operations (CyOps) by applying reinforcement and deep reinforcement learning (RL/DRL). The required RL training environment is particularly challenging, as it must balance the need for high-fidelity, best achieved through real network emulation, with the need for running large numbers of training episodes, best achieved using simulation. A unified training environment, namely the Cyber Gym for Intelligent Learning (CyGIL) is developed where an emulated CyGIL-E automatically generates a simulated CyGIL-S. From preliminary experimental results, CyGIL-S is capable to train agents in minutes compared with the days required in CyGIL-E. The agents trained in CyGIL-S are transferrable directly to CyGIL-E showing full decision proficiency in the emulated "real" network. Enabling offline RL, the CyGIL solution presents a promising direction towards sim-to-real for leveraging RL agents in real-world cyber networks.