Faheem Ullah

Faheem Ullah, Muhammad Ali Babar

Big Data Cyber Security Analytics (BDCA) systems use big data technologies (e.g., Apache Spark) to collect, store, and analyze a large volume of security event data for detecting cyber-attacks. The volume of digital data in general and security event data in specific is increasing exponentially. The velocity with which the security event data is generated and fed into a BDCA system is unpredictable. Therefore, a BDCA system should be highly scalable to deal with the unpredictable increase/decrease in the velocity of security event data. However, there has been little effort to investigate the scalability of BDCA systems to identify and exploit the sources of scalability improvement. In this paper, we first investigate the scalability of a Spark-based BDCA system with default Spark settings. we then identify Spark configuration parameters (e.g., execution memory) that can significantly impact the scalability of a BDCA system. Based on the identified parameters, we finally propose a parameter-driven adaptation approach, SCALER, for optimizing a system's scalability. We have conducted a set of experiments by implementing a Spark-based BDCA system on a large-scale OpenStack cluster. We ran our experiments with four security datasets. We have found that (i) a BDCA system with default Spark configuration parameters deviates from ideal scalability by 59.5% (ii) 9 out of 11 studied Spark configuration parameters significantly impact scalability (iii) SCALER improves the BDCA system's scalability by 20.8% compared to the scalability with default Spark parameter setting. The findings of our study highlight the importance of exploring the parameter space of the underlying big data framework (e.g., Apache Spark) for scalable cyber security analytics.

Bushra Sabir, Faheem Ullah, M. Ali Babar et al.

Context: Research at the intersection of cybersecurity, Machine Learning (ML), and Software Engineering (SE) has recently taken significant steps in proposing countermeasures for detecting sophisticated data exfiltration attacks. It is important to systematically review and synthesize the ML-based data exfiltration countermeasures for building a body of knowledge on this important topic. Objective: This paper aims at systematically reviewing ML-based data exfiltration countermeasures to identify and classify ML approaches, feature engineering techniques, evaluation datasets, and performance metrics used for these countermeasures. This review also aims at identifying gaps in research on ML-based data exfiltration countermeasures. Method: We used a Systematic Literature Review (SLR) method to select and review {92} papers. Results: The review has enabled us to (a) classify the ML approaches used in the countermeasures into data-driven, and behaviour-driven approaches, (b) categorize features into six types: behavioural, content-based, statistical, syntactical, spatial and temporal, (c) classify the evaluation datasets into simulated, synthesized, and real datasets and (d) identify 11 performance measures used by these studies. Conclusion: We conclude that: (i) the integration of data-driven and behaviour-driven approaches should be explored; (ii) There is a need of developing high quality and large size evaluation datasets; (iii) Incremental ML model training should be incorporated in countermeasures; (iv) resilience to adversarial learning should be considered and explored during the development of countermeasures to avoid poisoning attacks; and (v) the use of automated feature engineering should be encouraged for efficiently detecting data exfiltration attacks.

Faheem Ullah, M. Ali Babar

Context: Big Data Cybersecurity Analytics is aimed at protecting networks, computers, and data from unauthorized access by analysing security event data using big data tools and technologies. Whilst a plethora of Big Data Cybersecurity Analytic Systems have been reported in the literature, there is a lack of a systematic and comprehensive review of the literature from an architectural perspective. Objective: This paper reports a systematic review aimed at identifying the most frequently reported quality attributes and architectural tactics for Big Data Cybersecurity Analytic Systems. Method: We used Systematic Literature Review (SLR) method for reviewing 74 primary studies selected using well-defined criteria. Results: Our findings are twofold: (i) identification of 12 most frequently reported quality attributes and the justification for their significance for Big Data Cybersecurity Analytic Systems; and (ii) identification and codification of 17 architectural tactics for addressing the quality attributes that are commonly associated with Big Data Cybersecurity Analytic systems. The identified tactics include six performance tactics, four accuracy tactics, two scalability tactics, three reliability tactics, and one security and usability tactic each. Conclusion: Our findings have revealed that (a) despite the significance of interoperability, modifiability, adaptability, generality, stealthiness, and privacy assurance, these quality attributes lack explicit architectural support in the literature (b) empirical investigation is required to evaluate the impact of codified architectural tactics (c) a good deal of research effort should be invested to explore the trade-offs and dependencies among the identified tactics and (d) there is a general lack of effective collaboration between academia and industry for supporting the field of Big Data Cybersecurity Analytic Systems.