Towards Production-Worthy Simulation for Autonomous Cyber Operations
This work addresses the need for more realistic simulations in cybersecurity for training reinforcement learning agents, but it is incremental as it builds on an existing environment.
The study tackled the problem of improving simulation environments for Autonomous Cyber Operations by extending CybORG's Cage Challenge 2 with new actions and modifying reward signals and feature spaces, resulting in maintained ability to generate informative training signals for RL agents like DQN and PPO.
Simulated environments have proven invaluable in Autonomous Cyber Operations (ACO) where Reinforcement Learning (RL) agents can be trained without the computational overhead of emulation. These environments must accurately represent cybersecurity scenarios while producing the necessary signals to support RL training. In this study, we present a framework where we first extend CybORG's Cage Challenge 2 environment by implementing three new actions: Patch, Isolate, and Unisolate, to better represent the capabilities available to human operators in real-world settings. We then propose a design for agent development where we modify the reward signals and the agent's feature space to enhance training performance. To validate these modifications, we train DQN and PPO agents in the updated environment. Our study demonstrates that CybORG can be extended with additional realistic functionality, while maintaining its ability to generate informative training signals for RL agents.