Filipo Sharevski

Emma Pieroni, Peter Jachim, Nathaniel Jachim et al.

This paper evaluates Parler, the controversial social media platform, from two seemingly orthogonal perspectives: UX design perspective and data science. UX design researchers explore how users react to the interface/content of their social media feeds; Data science researchers analyze the misinformation flow in these feeds to detect alternative narratives and state-sponsored disinformation campaigns. We took a critical look into the intersection of these approaches to understand how Parler's interface itself is conductive to the flow of misinformation and the perception of "free speech" among its audience. Parler drew widespread attention leading up to and after the 2020 U.S. elections as the "alternative" place for free speech, as a reaction to other mainstream social media platform which actively engaged in labeling misinformation with content warnings. Because platforms like Parler are disruptive to the social media landscape, we believe the evaluation uniquely uncovers the platform's conductivity to the spread of misinformation.

Filipo Sharevski, Anna Slowinski, Peter Jachim et al.

In this paper, we analyzed the perceived accuracy of COVID-19 vaccine information spoken back by Amazon Alexa. Unlike social media, Amazon Alexa doesn't apply soft moderation to unverified content, allowing for use of third-party malicious skills to arbitrarily phrase COVID-19 vaccine information. The results from a 210-participant study suggest that a third-party malicious skill could successful reduce the perceived accuracy among the users of information as to who gets the vaccine first, vaccine testing, and the side effects of the vaccine. We also found that the vaccine-hesitant participants are drawn to pessimistically rephrased Alexa responses focused on the downsides of the mass immunization. We discuss solutions for soft moderation against misperception-inducing or altogether COVID-19 misinformation malicious third-party skills.

Donald Gover, Filipo Sharevski

In this paper, we analyzed the perceived accuracy of COVID-19 vaccine Tweets when they were spoken back by a third-party Amazon Alexa skill. We mimicked the soft moderation that Twitter applies to COVID-19 misinformation content in both forms of warning covers and warning tags to investigate whether the third-party skill could affect how and when users heed these warnings. The results from a 304-participant study suggest that the spoken back warning covers may not work as intended, even when converted from text to speech. We controlled for COVID-19 vaccination hesitancy and political leanings and found that the vaccination hesitant Alexa users ignored any type of warning as long as the Tweets align with their personal beliefs. The politically independent users trusted Alexa less then their politically-laden counterparts and that helped them accurately perceiving truthful COVID-19 information. We discuss soft moderation adaptations for voice assistants to achieve the intended effect of curbing COVID-19 misinformation.

Peter Jachim, Filipo Sharevski, Emma Pieroni

This paper presents Out-of-Context Summarizer, a tool that takes arbitrary public news articles out of context by summarizing them to coherently fit either a liberal- or conservative-leaning agenda. The Out-of-Context Summarizer also suggests hashtag keywords to bolster the polarization of the summary, in case one is inclined to take it to Twitter, Parler or other platforms for trolling. Out-of-Context Summarizer achieved 79% precision and 99% recall when summarizing COVID-19 articles, 93% precision and 93% recall when summarizing politically-centered articles, and 87% precision and 88% recall when taking liberally-biased articles out of context. Summarizing valid sources instead of synthesizing fake text, the Out-of-Context Summarizer could fairly pass the "adversarial disclosure" test, but we didn't take this easy route in our paper. Instead, we used the Out-of-Context Summarizer to push the debate of potential misuse of automated text generation beyond the boilerplate text of responsible disclosure of adversarial language models.

Filipo Sharevski, Raniem Alsaadi, Peter Jachim et al.

Twitter, prompted by the rapid spread of alternative narratives, started actively warning users about the spread of COVID-19 misinformation. This form of soft moderation comes in two forms: as a warning cover before the Tweet is displayed to the user and as a warning tag below the Tweet. This study investigates how each of the soft moderation forms affects the perceived accuracy of COVID-19 vaccine misinformation on Twitter. The results suggest that the warning covers work, but not the tags, in reducing the perception of accuracy of COVID-19 vaccine misinformation on Twitter. "Belief echoes" do exist among Twitter users, unfettered by any warning labels, in relationship to the perceived safety and efficacy of the COVID-19 vaccine as well as the vaccination hesitancy for themselves and their children. The implications of these results are discussed in the context of usable security affordances for combating misinformation on social media.

Peter Jachim, Filipo Sharevski, Emma Pieroni

This paper presents TrollHunter2020, a real-time detection mechanism we used to hunt for trolling narratives on Twitter during the 2020 U.S. elections. Trolling narratives form on Twitter as alternative explanations of polarizing events like the 2020 U.S. elections with the goal to conduct information operations or provoke emotional response. Detecting trolling narratives thus is an imperative step to preserve constructive discourse on Twitter and remove an influx of misinformation. Using existing techniques, this takes time and a wealth of data, which, in a rapidly changing election cycle with high stakes, might not be available. To overcome this limitation, we developed TrollHunter2020 to hunt for trolls in real-time with several dozens of trending Twitter topics and hashtags corresponding to the candidates' debates, the election night, and the election aftermath. TrollHunter2020 collects trending data and utilizes a correspondence analysis to detect meaningful relationships between the top nouns and verbs used in constructing trolling narratives while they emerge on Twitter. Our results suggest that the TrollHunter2020 indeed captures the emerging trolling narratives in a very early stage of an unfolding polarizing event. We discuss the utility of TrollHunter2020 for early detection of information operations or trolling and the implications of its use in supporting a constrictive discourse on the platform around polarizing topics.

Peter Jachim, Filipo Sharevski, Paige Treebridge

This paper presents TrollHunter, an automated reasoning mechanism we used to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. Trolls, poised to disrupt the online discourse and spread disinformation, quickly seized the absence of a credible response to COVID-19 and created a COVID-19 infodemic by promulgating dubious content on Twitter. To counter the COVID-19 infodemic, the TrollHunter leverages a unique linguistic analysis of a multi-dimensional set of Twitter content features to detect whether or not a tweet was meant to troll. TrollHunter achieved 98.5% accuracy, 75.4% precision and 69.8% recall over a dataset of 1.3 million tweets. Without a final resolution of the pandemic in sight, it is unlikely that the trolls will go away, although they might be forced to evade automated hunting. To explore the plausibility of this strategy, we developed and tested an adversarial machine learning mechanism called TrollHunter-Evader. TrollHunter-Evader employs a Test Time Evasion (TTE) approach in a combination with a Markov chain-based mechanism to recycle originally trolling tweets. The recycled tweets were able to achieve a remarkable 40% decrease in the TrollHunter's ability to correctly identify trolling tweets. Because the COVID-19 infodemic could have a harmful impact on the COVID-19 pandemic, we provide an elaborate discussion about the implications of employing adversarial machine learning to evade Twitter troll hunts.

Filipo Sharevski, Peter Jachim

This paper presents an automated adversarial mechanism called WikipediaBot. WikipediaBot allows an adversary to create and control a bot infrastructure for the purpose of adversarial edits of Wikipedia articles. The WikipediaBot is a self-contained mechanism with modules for generating credentials for Wikipedia editors, bypassing login protections, and a production of contextually-relevant adversarial edits for target Wikipedia articles that evade conventional detection. The contextually-relevant adversarial edits are generated using an adversarial Markov chain that incorporates a linguistic manipulation attack known as MIM or malware-induced misperceptions. Because the nefarious use of WikipediaBot could result in harmful damages to the integrity of wide range of Wikipedia articles, we provide an elaborate discussion about the implications, detection, and defenses Wikipedia could employ to address the threat of automated adversarial manipulations and acts of Wikipedia vandalism.

Filipo Sharevski, Peter Jachim, Kevin Florek

Trolling and social bots have been proven as powerful tactics for manipulating the public opinion and sowing discord among Twitter users. This effort requires substantial content fabrication and account coordination to evade Twitter's detection of nefarious platform use. In this paper we explore an alternative tactic for covert social media interference by inducing misperceptions about genuine, non-trolling content from verified users. This tactic uses a malware that covertly manipulates targeted words, hashtags, and Twitter metrics before the genuine content is presented to a targeted user in a covert man-in-the-middle fashion. Early tests of the malware found that it is capable of achieving a similar goal as trolls and social bots, that is, silencing or provoking social media users to express their opinion in polarized debates on social media. Following this, we conducted experimental tests in controlled settings (N=315) where the malware covertly manipulated the perception in a Twitter debate on the risk of vaccines causing autism. The empirical results demonstrate that inducing misperception is an effective tactic to silence users on Twitter when debating polarizing issues like vaccines. We used the findings to propose a solution for countering the effect of the malware-induced misperception that could also be used against trolls and social bots on Twitter.

Filipo Sharevski, Paige Treebridge, Peter Jachim et al.

Social media trolling is a powerful tactic to manipulate public opinion on issues with a high moral component. Troll farms, as evidenced in the past, created fabricated content to provoke or silence people to share their opinion on social media during the US presidential election in 2016. In this paper, we introduce an alternate way of provoking or silencing social media discourse by manipulating how users perceive authentic content. This manipulation is performed by man-in-the-middle malware that covertly rearranges the linguistic content of an authentic social media post and comments. We call this attack Malware-Induced Misperception (MIM) because the goal is to socially engineer spiral-of-silence conditions on social media by inducing perception. We conducted experimental tests in controlled settings (N = 311) where a malware covertly altered selected words in a Facebook post about the freedom of political expression on college campuses. The empirical results (1) confirm the previous findings about the presence of the spiral-of-silence effect on social media; and (2) demonstrate that inducing misperception is an effective tactic to silence or provoke targeted users on Facebook to express their opinion on a polarizing political issue.

Filipo Sharevski, Paige Treebridge, Peter Jachim et al.

This paper reports the findings of a study where users (N=220) interacted with Malexa, Alexa's malicious twin. Malexa is an intelligent voice assistant with a simple and seemingly harmless third-party skill that delivers news briefings to users. The twist, however, is that Malexa covertly rewords these briefings to intentionally introduce misperception about the reported events. This covert rewording is referred to as a Malware-Induced Misperception (MIM) attack. It differs from squatting or invocation hijacking attacks in that it is focused on manipulating the "content" delivered through a third-party skill instead of the skill's "invocation logic." Malexa, in the study, reworded regulatory briefings to make a government response sound more accidental or lenient than the original news delivered by Alexa. The results show that users who interacted with Malexa perceived that the government was less friendly to working people and more in favor of big businesses. The results also show that Malexa is capable of inducing misperceptions regardless of the user's gender, political ideology or frequency of interaction with intelligent voice assistants. We discuss the implications in the context of using Malexa as a covert "influencer" in people's living or working environments.

Filipo Sharevski, Paige Treebridge, Jessica Westbrook

This paper presents a specific man-in-the-middle exploit: Ambient Tactical Deception (ATD) in online communication, realized via a malicious web browser extension. Extensions manipulate web content in unobtrusive ways as ambient intermediaries of the overall browsing experience. In our previous work, we demonstrated that it is possible to employ tactical deception by making covert changes in the text content of a web page, regardless of the source. In this work, we investigated the application of ATD in a web-based email discourse where the objective is to manipulate the interpersonal perception without the knowledge of the involved parties. We focus on web-based email text because it is asynchronous and usually revised for clarity and politeness. Previous research has demonstrated that people's perception of politeness in online communication is based on three factors: the degree of imposition, the power of the receiver over the sender, and the social distance between them. We interviewed participants about their perception of these factors to establish the plausibility of ATD for email discourse. The results indicate that by covertly altering the politeness strategy in an email, it is possible for an ATD attacker to manipulate the receiver's perception on all of the politeness factors. Our findings support the Brown and Levinson's politeness theory and Walther's hyperpersonal model of email communication.

Alan T. Sherman, Linda Oliva, Enis Golaszewski et al.

For two days in February 2018, 17 cybersecurity educators and professionals from government and industry met in a "hackathon" to refine existing draft multiple-choice test items, and to create new ones, for a Cybersecurity Concept Inventory (CCI) and Cybersecurity Curriculum Assessment (CCA) being developed as part of the Cybersecurity Assessment Tools (CATS) Project. We report on the results of the CATS Hackathon, discussing the methods we used to develop test items, highlighting the evolution of a sample test item through this process, and offering suggestions to others who may wish to organize similar hackathons. Each test item embodies a scenario, question stem, and five answer choices. During the Hackathon, participants organized into teams to (1) Generate new scenarios and question stems, (2) Extend CCI items into CCA items, and generate new answer choices for new scenarios and stems, and (3) Review and refine draft CCA test items. The CATS Project provides rigorous evidence-based instruments for assessing and evaluating educational practices; these instruments can help identify pedagogies and content that are effective in teaching cybersecurity. The CCI measures how well students understand basic concepts in cybersecurity---especially adversarial thinking---after a first course in the field. The CCA measures how well students understand core concepts after completing a full cybersecurity curriculum.

Adam Trowbridge, Jessica Westbrook, Filipo Sharevski

In this paper we argue, drawing from the perspectives of cybersecurity and social psychology, that Internet-based manipulation of an individual or group reality using ambient tactical deception is possible using only software and changing words in a web browser. We call this attack Ambient Tactical Deception (ATD). Ambient, in artificial intelligence, describes software that is "unobtrusive," and completely integrated into a user's life. Tactical deception is an information warfare term for the use of deception on an opposing force. We suggest that an ATD attack could change the sentiment of text in a web browser. This could alter the victim's perception of reality by providing disinformation. Within the limit of online communication, even a pause in replying to a text can affect how people perceive each other. The outcomes of an ATD attack could include alienation, upsetting a victim, and influencing their feelings about an election, a spouse, or a corporation.

Adam Trowbridge, Filipo Sharevski, Jessica Westbrook

This paper explores the factors and theory behind the user-centered research that is necessary to create a successful game-like prototype, and user experience, for malicious users in a cybersecurity context. We explore what is known about successful addictive design in the fields of video games and gambling to understand the allure of breaking into a system, and the joy of thwarting the security to reach a goal or a reward of data. Based on the malicious user research, game user research, and using the GameFlow framework, we propose a novel malicious user experience design approach

Filipo Sharevski, Adam Trowbridge, Jessica Westbrook

Training the future cybersecurity workforce to respond to emerging threats requires introduction of novel educational interventions into the cybersecurity curriculum. To be effective, these interventions have to incorporate trending knowledge from cybersecurity and other related domains while allowing for experiential learning through hands-on experimentation. To date, the traditional interdisciplinary approach for cybersecurity training has infused political science, law, economics or linguistics knowledge into the cybersecurity curriculum, allowing for limited experimentation. Cybersecurity students were left with little opportunity to acquire knowledge, skills, and abilities in domains outside of these. Also, students in outside majors had no options to get into cybersecurity. With this in mind, we developed an interdisciplinary course for experiential learning in the fields of cybersecurity and interaction design. The inaugural course teaches students from cybersecurity, user interaction design, and visual design the principles of designing for secure use - or secure design - and allows them to apply them for prototyping of Internet-of-Things (IoT) products for smart homes. This paper elaborates on the concepts of secure design and how our approach enhances the training of the future cybersecurity workforce.