Towards a More Systematic Approach to Secure Systems Design and Analysis
This work addresses the challenge of inconsistent security advice for software system designers, though it is incremental in nature.
The paper tackled the problem of variability in security risk assessments among experts by conducting an exercise with 39 cybersecurity professionals independently evaluating a realistic system scenario, and found that aggregating their opinions yields a coherent consensus view that can inform secure systems design decisions.
The task of designing secure software systems is fraught with uncertainty, as data on uncommon attacks is limited, costs are difficult to estimate, and technology and tools are continually changing. Consequently, experts may interpret the security risks posed to a system in different ways, leading to variation in assessment. This paper presents research into measuring the variability in decision making between security professionals, with the ultimate goal of improving the quality of security advice given to software system designers. A set of thirty nine cyber-security experts took part in an exercise in which they independently assessed a realistic system scenario. This study quantifies agreement in the opinions of experts, examines methods of aggregating opinions, and produces an assessment of attacks from ratings of their components. We show that when aggregated, a coherent consensus view of security emerges which can be used to inform decisions made during systems design.