Automated Security Response through Online Learning with Adaptive Conjectures
This work addresses automated security response for IT infrastructure, offering a novel approach to handle model misspecification and uncertainty, though it is incremental in adapting existing game theory and learning methods to a specific domain.
The paper tackles the problem of automated security response in IT infrastructure by modeling attacker-defender interactions as a partially observed, non-stationary game with probabilistic conjectures, and introduces Conjectural Online Learning (COL) to adapt strategies online, resulting in effective security strategies and faster convergence than current reinforcement learning techniques.
We study automated security response for an IT infrastructure and formulate the interaction between an attacker and a defender as a partially observed, non-stationary game. We relax the standard assumption that the game model is correctly specified and consider that each player has a probabilistic conjecture about the model, which may be misspecified in the sense that the true model has probability 0. This formulation allows us to capture uncertainty and misconception about the infrastructure and the intents of the players. To learn effective game strategies online, we design Conjectural Online Learning (COL), a novel method where a player iteratively adapts its conjecture using Bayesian learning and updates its strategy through rollout. We prove that the conjectures converge to best fits, and we provide a bound on the performance improvement that rollout enables with a conjectured model. To characterize the steady state of the game, we propose a variant of the Berk-Nash equilibrium. We present COL through an advanced persistent threat use case. Testbed evaluations show that COL produces effective security strategies that adapt to a changing environment. We also find that COL enables faster convergence than current reinforcement learning techniques.