10.4CRJan 29, 2025
Towards Supporting Penetration Testing Education with Large Language Models: an Evaluation and ComparisonMartin Nizon-Deladoeuille, Brynjólfur Stefánsson, Helmut Neukirchen et al.
Cybersecurity education is challenging and it is helpful for educators to understand Large Language Models' (LLMs') capabilities for supporting education. This study evaluates the effectiveness of LLMs in conducting a variety of penetration testing tasks. Fifteen representative tasks were selected to cover a comprehensive range of real-world scenarios. We evaluate the performance of 6 models (GPT-4o mini, GPT-4o, Gemini 1.5 Flash, Llama 3.1 405B, Mixtral 8x7B and WhiteRabbitNeo) upon the Metasploitable v3 Ubuntu image and OWASP WebGOAT. Our findings suggest that GPT-4o mini currently offers the most consistent support making it a valuable tool for educational purposes. However, its use in conjonction with WhiteRabbitNeo should be considered, because of its innovative approach to tool and command recommendations. This study underscores the need for continued research into optimising LLMs for complex, domain-specific tasks in cybersecurity education.
6.4SEApr 6, 2021
On Adaptive Fairness in Software SystemsAli Farahani, Liliana Pasquale, Amel Bennaceur et al.
Software systems are increasingly making decisions on behalf of humans, raising concerns about the fairness of such decisions. Such concerns are usually attributed to flaws in algorithmic design or biased data, but we argue that they are often the result of a lack of explicit specification of fairness requirements. However, such requirements are challenging to elicit, a problem exacerbated by increasingly dynamic environments in which software systems operate, as well as stakeholders' changing needs. Therefore, capturing all fairness requirements during the production of software is challenging, and is insufficient for addressing software changes post deployment. In this paper, we propose adaptive fairness as a means for maintaining the satisfaction of changing fairness requirements. We demonstrate how to combine requirements-driven and resource-driven adaptation in order to address variabilities in both fairness requirements and their associated resources. Using models for fairness requirements, resources, and their relations, we show how the approach can be used to provide systems owners and end-users with capabilities that reflect adaptive fairness behaviours at runtime. We demonstrate our approach using an example drawn from shopping experiences of citizens. We conclude with a discussion of open research challenges in the engineering of adaptive fairness in human-facing software systems.