Assessing the Effectiveness of Attack Detection at a Hackfest on Industrial Control Systems
This work addresses cybersecurity for industrial control systems, specifically water treatment, but is incremental as it evaluates an existing detection method in a hackfest setting.
The paper analyzes the first two instances of the SWaT Security Showdown hackfest, which tested attack detection methods on a real water treatment plant, and reports the performance of the Water Defense mechanism against cyber attacks.
A hackfest named SWaT Security Showdown (S3) has been organized consecutively for two years. S3 has enabled researchers and practitioners to assess the effectiveness of methods and products aimed at detecting cyber attacks launched in real-time on an operational water treatment plant, namely, Secure Water Treatment (SWaT). In S3 independent attack teams design and launch attacks on SWaT while defence teams protect the plant passively and raise alarms upon attack detection. Attack teams are scored according to how successful they are in performing attacks based on specific intents while the defense teams are scored based on the effectiveness of their methods to detect the attacks. This paper focuses on the first two instances of S3 and summarizes the benefits of hackfest and the performance of an attack detection mechanism, named Water Defense, that was exposed to attackers during S3.