Kashyap Thimmaraju

Souradip Nath, Chih-Yi Huang, Aditi Ganapathi et al.

Large language models (LLMs) have recently emerged as promising tools for augmenting Security Operations Center (SOC) workflows, with vendors increasingly marketing autonomous AI solutions for SOCs. However, there remains a limited empirical understanding of how such tools are used, perceived, and adopted by real-world security practitioners. To address this gap, we conduct a mixed-methods analysis of discussions in cybersecurity-focused forums to learn how a diverse group of practitioners use and perceive modern LLM tools for security operations. More specifically, we analyzed 892 posts between December 2022 and September 2025 from three cybersecurity-focused forums on Reddit, and, using a combination of qualitative coding and statistical analysis, examined how security practitioners discuss LLM tools across three dimensions: (1) their stated tools and use cases, (2) the perceived pros and cons of each tool across a set of critical factors, and (3) their adoption of such tools and the expected impacts on the cybersecurity industry and individual analysts. Overall, our findings reveal nuanced patterns in LLM tools adoption, highlighting independent use of LLMs for low-risk, productivity-oriented tasks, alongside active interest around enterprise-grade, security-focused LLM platforms. Although practitioners report meaningful gains in efficiency and effectiveness in LLM-assisted workflows, persistent issues with reliability, verification overheads, and security risks sharply constrain the autonomy granted to LLM tools. Based on these results, we also provide recommendations for developing and adopting LLM tools to ensure the security of organizations and the safety of cybersecurity practitioners.

Hayfa Dhabhi, Kashyap Thimmaraju

Large Language Models (LLMs) deploy safety mechanisms to prevent harmful outputs, yet these defenses remain vulnerable to adversarial prompts. While existing research demonstrates that jailbreak attacks succeed, it does not explain \textit{where} defenses fail or \textit{why}. To address this gap, we propose that LLM safety operates as a sequential pipeline with distinct checkpoints. We introduce the \textbf{Four-Checkpoint Framework}, which organizes safety mechanisms along two dimensions: processing stage (input vs.\ output) and detection level (literal vs.\ intent). This creates four checkpoints, CP1 through CP4, each representing a defensive layer that can be independently evaluated. We design 13 evasion techniques, each targeting a specific checkpoint, enabling controlled testing of individual defensive layers. Using this framework, we evaluate GPT-5, Claude Sonnet 4, and Gemini 2.5 Pro across 3,312 single-turn, black-box test cases. We employ an LLM-as-judge approach for response classification and introduce Weighted Attack Success Rate (WASR), a severity-adjusted metric that captures partial information leakage overlooked by binary evaluation. Our evaluation reveals clear patterns. Traditional Binary ASR reports 22.6\% attack success. However, WASR reveals 52.7\%, a 2.3$\times$ higher vulnerability. Output-stage defenses (CP3, CP4) prove weakest at 72--79\% WASR, while input-literal defenses (CP1) are strongest at 13\% WASR. Claude achieves the strongest safety (42.8\% WASR), followed by GPT-5 (55.9\%) and Gemini (59.5\%). These findings suggest that current defenses are strongest at input-literal checkpoints but remain vulnerable to intent-level manipulation and output-stage techniques. The Four-Checkpoint Framework provides a structured approach for identifying and addressing safety vulnerabilities in deployed systems.

Giuseppe Canale, Kashyap Thimmaraju

Large Language Models (LLMs) are rapidly transitioning from conversational assistants to autonomous agents embedded in critical organizational functions, including Security Operations Centers (SOCs), financial systems, and infrastructure management. Current adversarial testing paradigms focus predominantly on technical attack vectors: prompt injection, jailbreaking, and data exfiltration. We argue this focus is catastrophically incomplete. LLMs, trained on vast corpora of human-generated text, have inherited not merely human knowledge but human \textit{psychological architecture} -- including the pre-cognitive vulnerabilities that render humans susceptible to social engineering, authority manipulation, and affective exploitation. This paper presents the first systematic application of the Cybersecurity Psychology Framework (\cpf{}), a 100-indicator taxonomy of human psychological vulnerabilities, to non-human cognitive agents. We introduce the \textbf{Synthetic Psychometric Assessment Protocol} (\sysname{}), a methodology for converting \cpf{} indicators into adversarial scenarios targeting LLM decision-making. Our preliminary hypothesis testing across seven major LLM families reveals a disturbing pattern: while models demonstrate robust defenses against traditional jailbreaks, they exhibit critical susceptibility to authority-gradient manipulation, temporal pressure exploitation, and convergent-state attacks that mirror human cognitive failure modes. We term this phenomenon \textbf{Anthropomorphic Vulnerability Inheritance} (AVI) and propose that the security community must urgently develop ``psychological firewalls'' -- intervention mechanisms adapted from the Cybersecurity Psychology Intervention Framework (\cpif{}) -- to protect AI agents operating in adversarial environments.

Kashyap Thimmaraju, Julian Fietkau, Fatemeh Ganji

In this paper we describe our experience in designing and evaluating our graduate level computer security seminar course. In particular, our seminar is designed with two goals in mind. First, to instil critical thinking by teaching graduate students how to read, review and present scientific literature. Second, to learn about the state-of-the-art in computer security and privacy research by reviewing proceedings from one of the top four security and privacy conferences including IEEE Symposium on Security and Privacy (Oakland SP), USENIX Security, Network and Distributed System Security Symposium (NDSS) and ACM Conference on Computer and Communications Security (CCS). The course entails each student to i) choose a specific technical session from the most recent conference, ii) review and present three papers from the chosen session and iii) analyze the relationship between the chosen papers from the session. To evaluate the course, we designed a set of questions to understand the motivation and decisions behind the students' choices as well as to evaluate and improve the quality of the course. Our key insights from the evaluation are the following: The three most popular topics of interest were Privacy, Web Security and Authentication, ii) 33% of the students chose the sessions based on the title of papers and iii) when providing an encouraging environment, students enjoy and engage in discussions.

Kashyap Thimmaraju, Stefan Schmid

We revisit multi-tenant network virtualization in data centers, and make the case for tenant-specific virtual switches. In particular, tenant-specific virtual switches allow cloud providers to extend fine-grained billing (known, e.g., from serverless architectures) to the network, accounting not only for IO, but also CPU or energy. We sketch an architecture and present economical motivation and recent technological enablers. We also find that virtual switches today do not offer sufficient multi-tenancy and can introduce artificial performance bottlenecks, e.g., in load balancers. We conclude by discussing additional use cases for tentant-specific switches.

Kashyap Thimmaraju, Liron Schiff, Stefan Schmid

Today's routing protocols critically rely on the assumption that the underlying hardware is trusted. Given the increasing number of attacks on network devices, and recent reports on hardware backdoors this assumption has become questionable. Indeed, with the critical role computer networks play today, the contrast between our security assumptions and reality is problematic. This paper presents Software-Defined Adversarial Trajectory Sampling (SoftATS), an OpenFlow-based mechanism to efficiently monitor packet trajectories, also in the presence of non-cooperating or even adversarial switches or routers, e.g., containing hardware backdoors. Our approach is based on a secure, redundant and adaptive sample distribution scheme which allows us to provably detect adversarial switches or routers trying to reroute, mirror, drop, inject, or modify packets (i.e., header and/or payload). We evaluate the effectiveness of our approach in different adversarial settings, report on a proof-of-concept implementation, and provide a first evaluation of the performance overheads of such a scheme.