Nitesh Saxena

Cagri Arisoy, Anuradha Mandal, Nitesh Saxena

The spread of digital disinformation (aka "fake news") is arguably one of the most significant threats on the Internet which can cause individual and societal harm of large scales. The susceptibility to fake news attacks hinges on whether Internet users perceive a fake news article/snippet to be legitimate after reading it. In this paper, we attempt to garner an in-depth understanding of users' susceptibility to text-centric fake news attacks via a neuro-cognitive methodology. We investigate the neural underpinnings relevant to fake/real news through EEG. We run an experiment with human users to pursue a thorough investigation of users' perception and cognitive processing of fake/real news. We analyze the neural activity associated with the fake/real news detection task for different categories of news articles. Our results show there may be no statistically significant or automatically inferable differences in the way the human brain processes the fake vs. real news, while marked differences are observed when people are subject to (real/fake) news vs. resting state and even between some different categories of fake news. This neuro-cognitive finding may help to justify users' susceptibility to fake news attacks, as also confirmed from the behavioral analysis. In other words, the fake news articles may seem almost indistinguishable from the real news articles in both behavioral and neural domains. Our work serves to dissect the fundamental neural phenomena underlying fake news attacks and explains users' susceptibility to these attacks through the limits of human biology. We believe this could be a notable insight for the researchers and practitioners suggesting the human detection of fake news might be ineffective, which may also have an adverse impact on the design of automated detection approaches that crucially rely upon human labeling of text articles for building training models

Jimmy Dani, Brandon McCulloh, Nitesh Saxena

"Honeywords" have emerged as a promising defense mechanism for detecting data breaches and foiling offline dictionary attacks (ODA) by deceiving attackers with false passwords. In this paper, we propose PassFilter, a novel deep learning (DL) based attack framework, fundamental in its ability to identify passwords from a set of sweetwords associated with a user account, effectively challenging a variety of honeywords generation techniques (HGTs). The DL model in PassFilter is trained with a set of previously collected or adversarially generated passwords and honeywords, and carefully orchestrated to predict whether a sweetword is the password or a honeyword. Our model can compromise the security of state-of-the-art, heuristics-based, and representation learning-based HGTs proposed by Dionysiou et al. Specifically, our analysis with nine publicly available password datasets shows that PassFilter significantly outperforms the baseline random guessing success rate of 5%, achieving 6.10% to 52.78% on the 1st guessing attempt, considering 20 sweetwords per account. This success rate rapidly increases with additional login attempts before account lock-outs, often allowed on many real-world online services to maintain reasonable usability. For example, it ranges from 41.78% to 96.80% for five attempts, and from 72.87% to 99.00% for ten attempts, compared to 25% and 50% random guessing, respectively. We also examined PassFilter against general-purpose language models used for honeyword generation, like those proposed by Yu et al. These honeywords also proved vulnerable to our attack, with success rates of 14.19% for 1st guessing attempt, increasing to 30.23%, 41.70%, and 63.10% after 3rd, 5th, and 10th guessing attempts, respectively. Our findings demonstrate the effectiveness of DL model deployed in PassFilter in breaching state-of-the-art HGTs and compromising password security based on ODA.

Md Mojibur Rahman Redoy Akanda, Ahmed Tanvir Mahdad, Nitesh Saxena

In today's technology-driven world, web services have opened up new opportunities for blind and visually impaired people to interact independently. Securing interactions with these services is crucial; however, currently deployed authentication mainly concentrate on sighted users, overlooking the needs of the blind and visually impaired community. In this paper, we address this gap by investigating the security and accessibility aspects of these authentication when adopted by blind and visually impaired users. We model web authentication for such users as screen reader assisted authentication and introduce an evaluation framework called AWARE. Using AWARE, we then systematically assessed popular PC and smartphone-based screen readers against different authentication methods, including variants of 2FA and passwordless schemes, to simulate real-world scenarios. We analyzed these screen reader assisted authentication interactions with authentication methods in three settings: using a terminal (PC) with screen readers, a combination of the terminal (PC) and smartphone with screen readers, and smartphones with integrated screen readers. The results of our study underscore weaknesses in all of our observed screen reader assisted scenarios for real-life authentication methods. These weaknesses, encompassing specific accessibility issues caused by imprecise screen reader instructions, highlight vulnerability concerning observed scenarios for both real-world and research literature based attacks, including phishing, concurrency, fatigue, cross-service, and shoulder surfing. Broadly, our AWARE framework can be used by designers as a precursor to user studies which are typically time-consuming and tedious to perform, independently allowing to unfold security and accessibility problems early which designers can address prior to full-fledged user testing of more isolated issues.

Kalyan Nakka, Nitesh Saxena

The inherent risk of generating harmful and unsafe content by Large Language Models (LLMs), has highlighted the need for their safety alignment. Various techniques like supervised fine-tuning, reinforcement learning from human feedback, and red-teaming were developed for ensuring the safety alignment of LLMs. However, the robustness of these aligned LLMs is always challenged by adversarial attacks that exploit unexplored and underlying vulnerabilities of the safety alignment. In this paper, we develop a novel black-box jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage for jailbreaking aligned LLMs. This represents a new direction in jailbreaking by exploiting fundamental information representation of data as continuous bits, rather than leveraging prompt engineering or adversarial manipulations. Our evaluation of five state-of-the-art LLMs, namely GPT-4o, Gemini 1.5, Claude 3.5, Llama 3.1, and Mixtral, in adversarial perspective, revealed the capabilities of BitBypass in bypassing their safety alignment and tricking them into generating harmful and unsafe content. Further, we observed that BitBypass outperforms several state-of-the-art jailbreak attacks in terms of stealthiness and attack success. Overall, these results highlights the effectiveness and efficiency of BitBypass in jailbreaking these state-of-the-art LLMs.

Fei Zhao, Runlin Zhang, Chengcui Zhang et al.

We present MarkMatch, a retrieval system for detecting whether two paper ballot marks were filled by the same hand. Unlike the previous SOTA method BubbleSig, which used binary classification on isolated mark pairs, MarkMatch ranks stylistic similarity between a query mark and a mark in the database using contrastive learning. Our model is trained with a dense batch similarity matrix and a dual loss objective. Each sample is contrasted against many negatives within each batch, enabling the model to learn subtle handwriting difference and improve generalization under handwriting variation and visual noise, while diagonal supervision reinforces high confidence on true matches. The model achieves an F1 score of 0.943, surpassing BubbleSig's best performance. MarkMatch also integrates Segment Anything Model for flexible mark extraction via box- or point-based prompts. The system offers election auditors a practical tool for visual, non-biometric investigation of suspicious ballots.

Kalyan Nakka, Jimmy Dani, Ausmit Mondal et al.

The growing adoption of Large Language Models (LLMs) has influenced the development of their lighter counterparts-Small Language Models (SLMs)-to enable on-device deployment across smartphones and edge devices. These SLMs offer enhanced privacy, reduced latency, server-free functionality, and improved user experience. However, due to resource constraints of on-device environment, SLMs undergo size optimization through compression techniques like quantization, which can inadvertently introduce fairness, ethical and privacy risks. Critically, quantized SLMs may respond to harmful queries directly, without requiring adversarial manipulation, raising significant safety and trust concerns. To address this, we propose LiteLMGuard (LLMG), an on-device prompt guard that provides real-time, prompt-level defense for quantized SLMs. Additionally, our prompt guard is designed to be model-agnostic such that it can be seamlessly integrated with any SLM, operating independently of underlying architectures. Our LLMG formalizes prompt filtering as a deep learning (DL)-based prompt answerability classification task, leveraging semantic understanding to determine whether a query should be answered by any SLM. Using our curated dataset, Answerable-or-Not, we trained and fine-tuned several DL models and selected ELECTRA as the candidate, with 97.75% answerability classification accuracy. Our safety effectiveness evaluations demonstrate that LLMG defends against over 87% of harmful prompts, including both direct instruction and jailbreak attack strategies. We further showcase its ability to mitigate the Open Knowledge Attacks, where compromised SLMs provide unsafe responses without adversarial prompting. In terms of prompt filtering effectiveness, LLMG achieves near state-of-the-art filtering accuracy of 94%, with an average latency of 135 ms, incurring negligible overhead for users.

Kalyan Nakka, Jimmy Dani, Nitesh Saxena

In this paper, we present a very first study to investigate trust and ethical implications of on-device artificial intelligence (AI), focusing on small language models (SLMs) amenable for personal devices like smartphones. While on-device SLMs promise enhanced privacy, reduced latency, and improved user experience compared to cloud-based services, we posit that they might also introduce significant risks and vulnerabilities compared to their on-server counterparts. As part of our trust assessment study, we conduct a systematic evaluation of the state-of-the-art on-devices SLMs, contrasted to their on-server counterparts, based on a well-established trustworthiness measurement framework. Our results show on-device SLMs to be significantly less trustworthy, specifically demonstrating more stereotypical, unfair and privacy-breaching behavior. Informed by these findings, we then perform our ethics assessment study using a dataset of unethical questions, that depicts harmful scenarios. Our results illustrate the lacking ethical safeguards in on-device SLMs, emphasizing their capabilities of generating harmful content. Further, the broken safeguards and exploitable nature of on-device SLMs is demonstrated using potentially unethical vanilla prompts, to which the on-device SLMs answer with valid responses without any filters and without the need for any jailbreaking or prompt engineering. These responses can be abused for various harmful and unethical scenarios like: societal harm, illegal activities, hate, self-harm, exploitable phishing content and many others, all of which indicates the severe vulnerability and exploitability of these on-device SLMs.

Zengrui Liu, Umar Iqbal, Nitesh Saxena

Data protection regulations, such as GDPR and CCPA, require websites and embedded third-parties, especially advertisers, to seek user consent before they can collect and process user data. Only when the users opt in, can these entities collect, process, and share user data. Websites typically incorporate Consent Management Platforms (CMPs), such as OneTrust and CookieBot, to solicit and convey user consent to the embedded advertisers, with the expectation that the consent will be respected. However, neither the websites nor the regulators currently have any mechanism to audit advertisers' compliance with the user consent, i.e., to determine if advertisers indeed do not collect, process, and share user data when the user opts out. In this paper, we propose an auditing framework that leverages advertisers' bidding behavior to empirically assess the violations of data protection regulations. Using our framework, we conduct a measurement study to evaluate four of the most widely deployed CMPs, i.e., Didomi, Quantcast, OneTrust, and CookieBot, as well as advertiser-offered opt-out controls, i.e., National Advertising Initiative's opt-out, under GDPR and CCPA. Our results indicate that in many cases user data is unfortunately still being collected, processed, and shared even when users opt-out. We also find that some CMPs are better than the others at conveying user consent and that several ad platforms ignore user consent. Our results also indicate that advertiser-offered opt-out are equally ineffective at protecting user privacy.

Zengrui Liu, Prakash Shrestha, Nitesh Saxena

We present a simple yet potentially devastating and hard-to-detect threat, called Gummy Browsers, whereby the browser fingerprinting information can be collected and spoofed without the victim's awareness, thereby compromising the privacy and security of any application that uses browser fingerprinting. The idea is that attacker A first makes the user U connect to his website (or to a well-known site the attacker controls) and transparently collects the information from U that is used for fingerprinting purposes. Then, A orchestrates a browser on his own machine to replicate and transmit the same fingerprinting information when connecting to W, fooling W to think that U is the one requesting the service rather than A. This will allow the attacker to profile U and compromise U's privacy. We design and implement the Gummy Browsers attack using three orchestration methods based on script injection, browser settings and debugging tools, and script modification, that can successfully spoof a wide variety of fingerprinting features to mimic many different browsers (including mobile browsers and the Tor browser). We then evaluate the attack against two state-of-the-art browser fingerprinting systems, FPStalker and Panopticlick. Our results show that A can accurately match his own manipulated browser fingerprint with that of any targeted victim user U's fingerprint for a long period of time, without significantly affecting the tracking of U and when only collecting U's fingerprinting information only once. The TPR (true positive rate) for the tracking of the benign user in the presence of the attack is larger than 0.9 in most cases. The FPR (false positive rate) for the tracking of the attacker is also high, larger than 0.9 in all cases. We also argue that the attack can remain completely oblivious to the user and the website, thus making it extremely difficult to thwart in practice.

Kiavash Satvat, Maliheh Shirvanian, Nitesh Saxena

In this paper, we introduce PASSAT, a practical system to boost the security assurance delivered by the current cloud architecture without requiring any changes or cooperation from the cloud service providers. PASSAT is an application transparent to the cloud servers that allows users to securely and efficiently store and access their files stored on public cloud storage based on a single master password. Using a fast and light-weight XOR secret sharing scheme, PASSAT secret-shares users' files and distributes them among n publicly available cloud platforms. To access the files, PASSAT communicates with any k out of n cloud platforms to receive the shares and runs a secret-sharing reconstruction algorithm to recover the files. An attacker (insider or outsider) who compromises or colludes with less than k platforms cannot learn the user's files or modify the files stealthily. To authenticate the user to multiple cloud platforms, PASSAT crucially stores the authentication credentials, specific to each platform on a password manager, protected under the user's master password. Upon requesting access to files, the user enters the password to unlock the vault and fetches the authentication tokens using which PASSAT can interact with cloud storage. Our instantiation of PASSAT based on (2, 3)-XOR secret sharing of Kurihara et al., implemented with three popular storage providers, namely, Google Drive, Box, and Dropbox, confirms that our approach can efficiently enhance the confidentiality, integrity, and availability of the stored files with no changes on the servers.

Chen Wang, Cong Shi, Yingying Chen et al.

Due to the open nature of voice input, voice assistant (VA) systems (e.g., Google Home and Amazon Alexa) are under a high risk of sensitive information leakage (e.g., personal schedules and shopping accounts). Though the existing VA systems may employ voice features to identify users, they are still vulnerable to various acoustic attacks (e.g., impersonation, replay and hidden command attacks). In this work, we focus on the security issues of the emerging VA systems and aim to protect the users' highly sensitive information from these attacks. Towards this end, we propose a system, WearID, which uses an off-the-shelf wearable device (e.g., a smartwatch or bracelet) as a secure token to verify the user's voice commands to the VA system. In particular, WearID exploits the readily available motion sensors from most wearables to describe the command sound in vibration domain and check the received command sound across two domains (i.e., wearable's motion sensor vs. VA device's microphone) to ensure the sound is from the legitimate user.

S Abhishek Anand, Chen Wang, Jian Liu et al.

In this paper, we build a speech privacy attack that exploits speech reverberations generated from a smartphone's in-built loudspeaker captured via a zero-permission motion sensor (accelerometer). We design our attack Spearphone2, and demonstrate that speech reverberations from inbuilt loudspeakers, at an appropriate loudness, can impact the accelerometer, leaking sensitive information about the speech. In particular, we show that by exploiting the affected accelerometer readings and carefully selecting feature sets along with off-the-shelf machine learning techniques, Spearphone can successfully perform gender classification (accuracy over 90%) and speaker identification (accuracy over 80%) for any audio/video playback on the smartphone. Our results with testing the attack on a voice call and voice assistant response were also encouraging, showcasing the impact of the proposed attack. In addition, we perform speech recognition and speech reconstruction to extract more information about the eavesdropped speech to an extent. Our work brings to light a fundamental design vulnerability in many currently-deployed smartphones, which may put people's speech privacy at risk while using the smartphone in the loudspeaker mode during phone calls, media playback or voice assistant interactions.

Kiavash Satvat, Nitesh Saxena

Harm to the privacy of users through data leakage is not an unknown issue, however, it has not been studied in the context of the crash reporting system. Automatic Crash Reporting Systems (ACRS) are used by applications to report information about the errors happening during a software failure. Although crash reports are valuable to diagnose errors, they may contain users' sensitive information. In this paper, we study such a privacy leakage vis-a-vis browsers' crash reporting systems. As a case study, we mine a dataset consisting of crash reports collected over the period of six years. Our analysis shows the presence of more than 20,000 sessions and token IDs, 600 passwords, 9,000 email addresses, an enormous amount of contact information, and other sensitive data. Our analysis sheds light on an important security and privacy issue in the current state-of-the-art browser crash reporting systems. Further, we propose a hotfix to enhance users' privacy and security in ACRS by removing sensitive data from the crash report prior to submit the report to the server. Our proposed hotfix can be easily integrated into the current implementation of ACRS and has no impact on the process of fixing bugs while maintaining the reports' readability.

Maliheh Shirvanian, Nitesh Saxena, Jesvin James George

Many widely used Internet messaging and calling apps, such as WhatsApp, Viber, Telegram, and Signal, have deployed an end-to-end encryption functionality. To defeat potential MITM attackers against the key exchange protocol, the approach relies on users to perform a code verification task whereby each user must compare the code (a fingerprint of the cryptographic keys) computed by her app with the one computed by the other user's app and reject the session if the two do not match. In this paper, we study the security and usability of this human-centered code verification task for a setting where the end users are remotely located, and compare it as a baseline to a less frequent scenario where the users are in close proximity. We consider several variations of the code presentation and verification methods, incorporated into representative real-world apps, including codes encoded as numbers or images, displayed on the screen, and verbally spoken by the users. We perform a human factors study in a lab setting to quantify the security and usability of these different methods. Our study results expose key weaknesses in the security and usability of the code verification methods employed in the apps. First, we show that most code verification methods offer poor security (high false accepts) and low usability (high false rejects and low user experience ratings) in the remote setting. Second, we demonstrate that, security and usability under the remote code verification setting is significantly lower than that in the proximity setting. We attribute this result to the increased cognitive overhead associated with comparing the codes across two apps on the same device (remote setting) rather than across two devices (proximity setting). Overall, our work serves to highlight a serious vulnerability of Internet-based communication apps in the remote setting stemming from human errors.

Babins Shrestha, Manar Mohamed, Nitesh Saxena

Zero-interaction authentication (ZIA) refers to a form of user-transparent login mechanism using which a terminal (e.g., a desktop computer) can be unlocked by the mere proximity of an authentication token (e.g., a smartphone). Given its appealing usability, ZIA has already been deployed in many real-world applications. However, ZIA contains one major security weakness - unauthorized physical access to the token, e.g., during lunch-time or upon theft, allows the attacker to have unfettered access to the terminal. In this paper, we address this gaping vulnerability with ZIA systems by (un)locking the authentication token with the user's walking pattern as she approaches the terminal to access it. Since a user's walking or gait pattern is believed to be unique, only that user (no imposter) would be able to unlock the token to gain access to the terminal in a ZIA session. While walking-based biometrics schemes have been studied in prior literature for other application settings, our main novelty lies in the careful use of: (1) multiple sensors available on the current breed of devices (e.g., accelerometer, gyroscope and magnetometer), and (2) multiple devices carried by the user (in particular, an "in-pocket" smartphone and a "wrist-worn" smartwatch), that all capture unique facets of user's walking pattern. Our contributions are three-fold. First, we introduce, design and implement WUZIA ("Walk-Unlock ZIA"), a multi-modal walking biometrics approach tailored to enhance the security of ZIA systems (still with zero interaction). Second, we demonstrate that WUZIA offers a high degree of detection accuracy, based on multi-sensor and multi-device fusion. Third, we show that WUZIA can resist active attacks that attempt to mimic a user's walking pattern, especially when multiple devices are used.

Babins Shrestha, Nitesh Saxena, Hien Thi Thu Truong et al.

Contextual proximity detection (or, co-presence detection) is a promising approach to defend against relay attacks in many mobile authentication systems. We present a systematic assessment of co-presence detection in the presence of a context-manipulating attacker. First, we show that it is feasible to manipulate, consistently control and stabilize the readings of different acoustic and physical environment sensors (and even multiple sensors simultaneously) using low-cost, off-the-shelf equipment. Second, based on these capabilities, we show that an attacker who can manipulate the context gains a significant advantage in defeating context-based co-presence detection. For systems that use multiple sensors, we investigate two sensor fusion approaches based on machine learning techniques: features-fusion and decisions-fusion, and show that both are vulnerable to contextual attacks but the latter approach can be more resistant in some cases.

Manar Mohamed, Niharika Sachdeva, Michael Georgescu et al.

Existing captcha solutions on the Internet are a major source of user frustration. Game captchas are an interesting and, to date, little-studied approach claiming to make captcha solving a fun activity for the users. One broad form of such captchas -- called Dynamic Cognitive Game (DCG) captchas -- challenge the user to perform a game-like cognitive task interacting with a series of dynamic images. We pursue a comprehensive analysis of a representative category of DCG captchas. We formalize, design and implement such captchas, and dissect them across: (1) fully automated attacks, (2) human-solver relay attacks, and (3) usability. Our results suggest that the studied DCG captchas exhibit high usability and, unlike other known captchas, offer some resistance to relay attacks, but they are also vulnerable to our novel dictionary-based automated attack.

Haoyu Li, Di Ma, Nitesh Saxena et al.

In this paper, we introduce a lightweight permission enforcement approach - Tap-Wave-Rub (TWR) - for smartphone malware prevention. TWR is based on simple human gestures that are very quick and intuitive but less likely to be exhibited in users' daily activities. Presence or absence of such gestures, prior to accessing an application, can effectively inform the OS whether the access request is benign or malicious. Specifically, we present the design of two mechanisms: (1) accelerometer based phone tapping detection; and (2) proximity sensor based finger tapping, rubbing or hand waving detection. The first mechanism is geared for NFC applications, which usually require the user to tap her phone with another device. The second mechanism involves very simple gestures, i.e., tapping or rubbing a finger near the top of phone's screen or waving a hand close to the phone, and broadly appeals to many applications (e.g., SMS). In addition, we present the TWR-enhanced Android permission model, the prototypes implementing the underlying gesture recognition mechanisms, and a variety of novel experiments to evaluate these mechanisms. Our results suggest the proposed approach could be very effective for malware detection and prevention, with quite low false positives and false negatives, while imposing little to no additional burden on the users.