Using Cyber Terrain in Reinforcement Learning for Penetration Testing
This work tackles the problem of unrealistic attack simulations in cybersecurity for penetration testers, but it is incremental as it builds on existing RL methods with a new domain-specific adaptation.
The paper addresses the lack of realism in reinforcement learning for penetration testing by incorporating cyber terrain analysis from intelligence preparation of the battlefield into attack graphs, demonstrating this on an example with firewalls to improve realism.
Reinforcement learning (RL) has been applied to attack graphs for penetration testing, however, trained agents do not reflect reality because the attack graphs lack operational nuances typically captured within the intelligence preparation of the battlefield (IPB) that include notions of (cyber) terrain. In particular, current practice constructs attack graphs exclusively using the Common Vulnerability Scoring System (CVSS) and its components. We present methods for constructing attack graphs using notions from IPB on cyber terrain analysis of obstacles, avenues of approach, key terrain, observation and fields of fire, and cover and concealment. We demonstrate our methods on an example where firewalls are treated as obstacles and represented in (1) the reward space and (2) the state dynamics. We show that terrain analysis can be used to bring realism to attack graphs for RL.