Daniel Woods

Sayon Palit, Daniel Woods

Large Language Models (LLMs) are increasingly integrated into critical systems in industries like healthcare and finance. Users can often submit queries to LLM-enabled chatbots, some of which can enrich responses with information retrieved from internal databases storing sensitive data. This gives rise to a range of attacks in which a user submits a malicious query and the LLM-system outputs a response that creates harm to the owner, such as leaking internal data or creating legal liability by harming a third-party. While security tools are being developed to counter these threats, there is little formal evaluation of their effectiveness and usability. This study addresses this gap by conducting a thorough comparative analysis of LLM security tools. We identified 13 solutions (9 closed-source, 4 open-source), but only 7 were evaluated due to a lack of participation by proprietary model owners.To evaluate, we built a benchmark dataset of malicious prompts, and evaluate these tools performance against a baseline LLM model (ChatGPT-3.5-Turbo). Our results show that the baseline model has too many false positives to be used for this task. Lakera Guard and ProtectAI LLM Guard emerged as the best overall tools showcasing the tradeoff between usability and performance. The study concluded with recommendations for greater transparency among closed source providers, improved context-aware detections, enhanced open-source engagement, increased user awareness, and the adoption of more representative performance metrics.

Mustafa Abdallah, Daniel Woods, Parinaz Naghizadeh et al.

We model the behavioral biases of human decision-making in securing interdependent systems and show that such behavioral decision-making leads to a suboptimal pattern of resource allocation compared to non-behavioral (rational) decision-making. We provide empirical evidence for the existence of such behavioral bias model through a controlled subject study with 145 participants. We then propose three learning techniques for enhancing decision-making in multi-round setups. We illustrate the benefits of our decision-making model through multiple interdependent real-world systems and quantify the level of gain compared to the case in which the defenders are behavioral. We also show the benefit of our learning techniques against different attack models. We identify the effects of different system parameters on the degree of suboptimality of security outcomes due to behavioral decision-making.

Mustafa Abdallah, Daniel Woods, Parinaz Naghizadeh et al.

We study the security of large-scale cyber-physical systems (CPS) consisting of multiple interdependent subsystems, each managed by a different defender. Defenders invest their security budgets with the goal of thwarting the spread of cyber attacks to their critical assets. We model the security investment decisions made by the defenders as a security game. While prior work has used security games to analyze such scenarios, we propose behavioral security games, in which defenders exhibit characteristics of human decision making that have been identified in behavioral economics as representing typical human cognitive biases. This is important as many of the critical security decisions in our target class of systems are made by humans. We provide empirical evidence for our behavioral model through a controlled subject experiment. We then show that behavioral decision making leads to a suboptimal pattern of resource allocation compared to non-behavioral decision making. We illustrate the effects of behavioral decision making using two representative real-world interdependent CPS. In particular, we identify the effects of the defenders' security budget availability and distribution, the degree of interdependency among defenders, and collaborative defense strategies, on the degree of suboptimality of security outcomes due to behavioral decision making. In this context, the adverse effects of behavioral decision making are most severe with moderate defense budgets. Moreover, the impact of behavioral suboptimal decision making is magnified as the degree of the interdependency between subnetworks belonging to different defenders increases. We also observe that selfish defense decisions together with behavioral decisions significantly increase security risk.