Rouzbeh Behnia

Rouzbeh Behnia, Mohamamdreza Ebrahimi, Jason Pacheco et al.

Pre-trained Large Language Models (LLMs) are an integral part of modern AI that have led to breakthrough performances in complex AI tasks. Major AI companies with expensive infrastructures are able to develop and train these large models with billions and millions of parameters from scratch. Third parties, researchers, and practitioners are increasingly adopting these pre-trained models and fine-tuning them on their private data to accomplish their downstream AI tasks. However, it has been shown that an adversary can extract/reconstruct the exact training samples from these LLMs, which can lead to revealing personally identifiable information. The issue has raised deep concerns about the privacy of LLMs. Differential privacy (DP) provides a rigorous framework that allows adding noise in the process of training or fine-tuning LLMs such that extracting the training data becomes infeasible (i.e., with a cryptographically small success probability). While the theoretical privacy guarantees offered in most extant studies assume learning models from scratch through many training iterations in an asymptotic setting, this assumption does not hold in fine-tuning scenarios in which the number of training iterations is significantly smaller. To address the gap, we present \ewtune, a DP framework for fine-tuning LLMs based on Edgeworth accountant with finite-sample privacy guarantees. Our results across four well-established natural language understanding (NLU) tasks show that while \ewtune~adds privacy guarantees to LLM fine-tuning process, it directly contributes to decreasing the induced noise to up to 5.6\% and improves the state-of-the-art LLMs performance by up to 1.1\% across all NLU tasks. We have open-sourced our implementations for wide adoption and public testing purposes.

Yiwei Zhang, Jeremiah Birrell, Reza Ebrahimi et al.

Large language models (LLMs) remain vulnerable to adversarial prompting despite advances in alignment and safety, often exhibiting harmful behaviors under novel attack strategies. While adversarial training can improve robustness, existing approaches are computationally expensive and difficult to scale. Recent continuous adversarial training methods, such as Continuous adversarial training (CAT) and Continuous Adversarial Preference Optimization (CAPO), address this challenge by leveraging gradient-based perturbations in the embedding space, enabling more efficient and expressive attacks. Building on this paradigm, we propose WARDEN, a distributionally robust adversarial training framework for LLMs that dynamically reweights adversarial examples through an f -divergence ambiguity set around the empirical training distribution. Our method optimizes the worst-case adversarial loss within a divergence ball around the empirical data distribution, automatically emphasizing harder adversarial examples. Using the convex dual formulation, the objective reduces to a log-sum-exp form under the KL divergence, with a dynamical parameter controlling the strength of reweighting. This study leads to a new class of information-theoretic objectives that significantly reduce attack success rates while maintaining model utility. Across multiple LLMs and attack settings, WARDEN substantially reduces attack success rates with computational and utility costs comparable to CAT-, CAPO-, and MixAT-based baselines, making it a practical approach for scalable robust alignment.

Jeremiah Birrell, Reza Ebrahimi, Rouzbeh Behnia et al.

Differentially private stochastic gradient descent (DP-SGD) has been instrumental in privately training deep learning models by providing a framework to control and track the privacy loss incurred during training. At the core of this computation lies a subsampling method that uses a privacy amplification lemma to enhance the privacy guarantees provided by the additive noise. Fixed size subsampling is appealing for its constant memory usage, unlike the variable sized minibatches in Poisson subsampling. It is also of interest in addressing class imbalance and federated learning. However, the current computable guarantees for fixed-size subsampling are not tight and do not consider both add/remove and replace-one adjacency relationships. We present a new and holistic R{é}nyi differential privacy (RDP) accountant for DP-SGD with fixed-size subsampling without replacement (FSwoR) and with replacement (FSwR). For FSwoR we consider both add/remove and replace-one adjacency. Our FSwoR results improves on the best current computable bound by a factor of $4$. We also show for the first time that the widely-used Poisson subsampling and FSwoR with replace-one adjacency have the same privacy to leading order in the sampling probability. Accordingly, our work suggests that FSwoR is often preferable to Poisson subsampling due to constant memory usage. Our FSwR accountant includes explicit non-asymptotic upper and lower bounds and, to the authors' knowledge, is the first such analysis of fixed-size RDP with replacement for DP-SGD. We analytically and empirically compare fixed size and Poisson subsampling, and show that DP-SGD gradients in a fixed-size subsampling regime exhibit lower variance in practice in addition to memory usage benefits.

Rouzbeh Behnia, Attila A. Yavuz, Muslum Ozgur Ozmen et al.

Certificates ensure the authenticity of users' public keys, however their overhead (e.g., certificate chains) might be too costly for some IoT systems like aerial drones. Certificate-free cryptosystems, like identity-based and certificateless systems, lift the burden of certificates and could be a suitable alternative for such IoTs. However, despite their merits, there is a research gap in achieving compatible identity-based and certificateless systems to allow users from different domains (identity-based or certificateless) to communicate seamlessly. Moreover, more efficient constructions can enable their adoption in resource-limited IoTs. In this work, we propose new identity-based and certificateless cryptosystems that provide such compatibility and efficiency. This feature is beneficial for heterogeneous IoT settings (e.g., commercial aerial drones), where different levels of trust/control is assumed on the trusted third party. Our schemes are more communication efficient than their public key based counterparts, as they do not need certificate processing. Our experimental analysis on both commodity and embedded IoT devices show that, only with the cost of having a larger system public key, our cryptosystems are more computation and communication efficient than their certificate-free counterparts. We prove the security of our schemes (in the random oracle model) and open-source our cryptographic framework for public testing/adoption.

Muslum Ozgur Ozmen, Rouzbeh Behnia, Attila A. Yavuz

Internet of Drones (IoD) is expected to play a central role in many civilian and military applications, that require sensitive and mission-critical information to be processed. It is therefore vital to ensure the security and privacy of IoD. However, unlike traditional networks, IoD has a broader attack surface and is highly energy-constrained, which hinder the direct adoption of standard cryptographic protocols for IoD. We propose an energy-efficient cryptographic framework (namely IoD-Crypt), which can potentially meet the requirements of battery-limited IoD. Specifically, IoD-Crypt utilizes special precomputation techniques and self-certified primitives to gain significant computation and communication efficiency over the standard public key cryptography (PKC) suites. Our integrations and optimizations are broadly applicable to key exchange, digital signature and public key encryption schemes that encompass generic applications of PKC in IoD. We prove that IoD-Crypt is secure in the random oracle model. We fully implemented IoD-Crypt on two common drone processors, namely 8-bit AVR and 32-bit ARM, and conducted an in-depth energy analysis. Our experiments (on both platforms) showed that IoD-Crypt offers up to 48x less energy consumption compared to standard techniques. We have open-sourced our implementations for wide adoption and public testing purposes.

Muslum Ozgur Ozmen, Attila A. Yavuz, Rouzbeh Behnia

Authentication is vital for the Internet of Things (IoT) applications involving sensitive data (e.g., medical and financial systems). Digital signatures offer scalable authentication with non-repudiation and public verifiability, which are necessary for auditing and dispute resolution in such IoT applications. However, digital signatures have been shown to be highly costly for low-end IoT devices, especially when embedded devices (e.g., medical implants) must operate without a battery replacement for a long time. We propose an Energy-aware Signature for Embedded Medical devices (ESEM) that achieves near-optimal signer efficiency. ESEM signature generation does not require any costly operations (e.g., elliptic curve (EC) scalar multiplication/addition), but only a small constant-number of pseudo-random function calls, additions, and a single modular multiplication. ESEM has the smallest signature size among its EC-based counterparts with an identical private key size. We achieve this by eliminating the use of the ephemeral public key (i.e, commitment) in Schnorr-type signatures from the signing via a distributed construction at the verifier without interaction with the signer while permitting a constant-size public key. We proved that ESEM is secure (in random oracle model), and fully implemented it on an 8-bit AVR microcontroller that is commonly used in medical devices. Our experiments showed that ESEM achieves 8.4x higher energy efficiency over its closest counterpart while offering a smaller signature and code size. Hence, ESEM can be suitable for deployment on resource limited embedded devices in IoT. We open-sourced our software for public testing and wide-adoption.

Rouzbeh Behnia, Muslum Ozgur Ozmen, Attila A. Yavuz

Efficient authentication is vital for IoT applications with stringent minimum-delay requirements (e.g., energy delivery systems). This requirement becomes even more crucial when the IoT devices are battery-powered, like small aerial drones, and the efficiency of authentication directly translates to more operation time. Although some fast authentication techniques have been proposed, some of them might not fully meet the needs of the emerging delay-aware IoT. In this paper, we propose a new signature scheme called ARIS that pushes the limits of the existing digital signatures, wherein commodity hardware can verify 83,333 signatures per second. ARIS also enables the fastest signature generation along with the lowest energy consumption and end-to-end delay among its counterparts. These significant computational advantages come with a larger storage requirement, which is a highly favorable trade-off for some critical delay-aware applications. These desirable features are achieved by harnessing message encoding with cover-free families and special elliptic curve based one-way function. We prove the security of ARIS under the hardness of the elliptic curve discrete logarithm problem in the random oracle model. We provide an open-sourced implementation of ARIS on commodity hardware and 8-bit AVR microcontroller for public testing and verification.

Yiwei Zhang, Rouzbeh Behnia, Attila A. Yavuz et al.

Federated learning enables the collaborative learning of a global model on diverse data, preserving data locality and eliminating the need to transfer user data to a central server. However, data privacy remains vulnerable, as attacks can target user training data by exploiting the updates sent by users during each learning iteration. Secure aggregation protocols are designed to mask/encrypt user updates and enable a central server to aggregate the masked information. MicroSecAgg (PoPETS 2024) proposes a single server secure aggregation protocol that aims to mitigate the high communication complexity of the existing approaches by enabling a one-time setup of the secret to be re-used in multiple training iterations. In this paper, we identify a security flaw in the MicroSecAgg that undermines its privacy guarantees. We detail the security flaw and our attack, demonstrating how an adversary can exploit predictable masking values to compromise user privacy. Our findings highlight the critical need for enhanced security measures in secure aggregation protocols, particularly the implementation of dynamic and unpredictable masking strategies. We propose potential countermeasures to mitigate these vulnerabilities and ensure robust privacy protection in the secure aggregation frameworks.

Yuncong Yang, Xiao Han, Yidong Chai et al.

Recent privacy regulations (e.g., GDPR) grant data subjects the `Right to Be Forgotten' (RTBF) and mandate companies to fulfill data erasure requests from data subjects. However, companies encounter great challenges in complying with the RTBF regulations, particularly when asked to erase specific training data from their well-trained predictive models. While researchers have introduced machine unlearning methods aimed at fast data erasure, these approaches often overlook maintaining model performance (e.g., accuracy), which can lead to financial losses and non-compliance with RTBF obligations. This work develops a holistic machine learning-to-unlearning framework, called Ensemble-based iTerative Information Distillation (ETID), to achieve efficient data erasure while preserving the business value of predictive models. ETID incorporates a new ensemble learning method to build an accurate predictive model that can facilitate handling data erasure requests. ETID also introduces an innovative distillation-based unlearning method tailored to the constructed ensemble model to enable efficient and effective data erasure. Extensive experiments demonstrate that ETID outperforms various state-of-the-art methods and can deliver high-quality unlearned models with efficiency. We also highlight ETID's potential as a crucial tool for fostering a legitimate and thriving market for data and predictive services.

Atousa Arzanipour, Rouzbeh Behnia, Reza Ebrahimi et al.

Retrieval-Augmented Generation (RAG) is an emerging approach in natural language processing that combines large language models (LLMs) with external document retrieval to produce more accurate and grounded responses. While RAG has shown strong potential in reducing hallucinations and improving factual consistency, it also introduces new privacy and security challenges that differ from those faced by traditional LLMs. Existing research has demonstrated that LLMs can leak sensitive information through training data memorization or adversarial prompts, and RAG systems inherit many of these vulnerabilities. At the same time, reliance of RAG on an external knowledge base opens new attack surfaces, including the potential for leaking information about the presence or content of retrieved documents, or for injecting malicious content to manipulate model behavior. Despite these risks, there is currently no formal framework that defines the threat landscape for RAG systems. In this paper, we address a critical gap in the literature by proposing, to the best of our knowledge, the first formal threat model for retrieval-RAG systems. We introduce a structured taxonomy of adversary types based on their access to model components and data, and we formally define key threat vectors such as document-level membership inference and data poisoning, which pose serious privacy and integrity risks in real-world deployments. By establishing formal definitions and attack models, our work lays the foundation for a more rigorous and principled understanding of privacy and security in RAG systems.

Kasra Ahmadi, Rouzbeh Behnia, Reza Ebrahimi et al.

Federated learning (FL) enhances privacy by keeping user data on local devices. However, emerging attacks have demonstrated that the updates shared by users during training can reveal significant information about their data. This has greatly thwart the adoption of FL methods for training robust AI models in sensitive applications. Differential Privacy (DP) is considered the gold standard for safeguarding user data. However, DP guarantees are highly conservative, providing worst-case privacy guarantees. This can result in overestimating privacy needs, which may compromise the model's accuracy. Additionally, interpretations of these privacy guarantees have proven to be challenging in different contexts. This is further exacerbated when other factors, such as the number of training iterations, data distribution, and specific application requirements, can add further complexity to this problem. In this work, we proposed a framework that integrates a human entity as a privacy practitioner to determine an optimal trade-off between the model's privacy and utility. Our framework is the first to address the variable memory requirement of existing DP methods in FL settings, where resource-limited devices (e.g., cell phones) can participate. To support such settings, we adopt a recent DP method with fixed memory usage to ensure scalable private FL. We evaluated our proposed framework by fine-tuning a BERT-based LLM model using the GLUE dataset (a common approach in literature), leveraging the new accountant, and employing diverse data partitioning strategies to mimic real-world conditions. As a result, we achieved stable memory usage, with an average accuracy reduction of 1.33% for $ε= 10$ and 1.9% for $ε= 6$, when compared to the state-of-the-art DP accountant which does not support fixed memory usage.

Varol Kayhan, Shivendu Shivendu, Rouzbeh Behnia et al.

Threat hunting is sifting through system logs to detect malicious activities that might have bypassed existing security measures. It can be performed in several ways, one of which is based on detecting anomalies. We propose an unsupervised framework, called continuous bag-of-terms-and-time (CBoTT), and publish its application programming interface (API) to help researchers and cybersecurity analysts perform anomaly-based threat hunting among SIEM logs geared toward process auditing on endpoint devices. Analyses show that our framework consistently outperforms benchmark approaches. When logs are sorted by likelihood of being an anomaly (from most likely to least), our approach identifies anomalies at higher percentiles (between 1.82-6.46) while benchmark approaches identify the same anomalies at lower percentiles (between 3.25-80.92). This framework can be used by other researchers to conduct benchmark analyses and cybersecurity analysts to find anomalies in SIEM logs.