Jeffrey Pawlick

Tao Zhang, Linan Huang, Jeffrey Pawlick et al.

Deception is a technique to mislead human or computer systems by manipulating beliefs and information. For the applications of cyber deception, non-cooperative games become a natural choice of models to capture the adversarial interactions between the players and quantitatively characterizes the conflicting incentives and strategic responses. In this chapter, we provide an overview of deception games in three different environments and extend the baseline signaling game models to include evidence through side-channel knowledge acquisition to capture the information asymmetry, dynamics, and strategic behaviors of deception. We analyze the deception in binary information space based on a signaling game framework with a detector that gives off probabilistic evidence of the deception when the sender acts deceptively. We then focus on a class of continuous one-dimensional information space and take into account the cost of deception in the signaling game. We finally explore the multi-stage incomplete-information Bayesian game model for defensive deception for advanced persistent threats (APTs). We use the perfect Bayesian Nash equilibrium (PBNE) as the solution concept for the deception games and analyze the strategic equilibrium behaviors for both the deceivers and the deceivees.

Jeffrey Pawlick, Juntao Chen, Quanyan Zhu

The cloud-enabled Internet of controlled things (IoCT) envisions a network of sensors, controllers, and actuators connected through a local cloud in order to intelligently control physical devices. Because cloud services are vulnerable to advanced persistent threats (APTs), each device in the IoCT must strategically decide whether to trust cloud services that may be compromised. In this paper, we present iSTRICT, an interdependent strategic trust mechanism for the cloud-enabled IoCT. iSTRICT is composed of three interdependent layers. In the cloud layer, iSTRICT uses FlipIt games to conceptualize APTs. In the communication layer, it captures the interaction between devices and the cloud using signaling games. In the physical layer, iSTRICT uses optimal control to quantify the utilities in the higher level games. Best response dynamics link the three layers in an overall "game-of-games," for which the outcome is captured by a concept called Gestalt Nash equilibrium (GNE). We prove the existence of a GNE under a set of natural assumptions and develop an adaptive algorithm to iteratively compute the equilibrium. Finally, we apply iSTRICT to trust management for autonomous vehicles that rely on measurements from remote sources. We show that strategic trust in the communication layer achieves a worst-case probability of compromise for any attack and defense costs in the cyber layer.

Jeffrey Pawlick, Edward Colbert, Quanyan Zhu

Deception plays critical roles in economics and technology, especially in emerging interactions in cyberspace. Holistic models of deception are needed in order to analyze interactions and to design mechanisms that improve them. Game theory provides such models. In particular, existing work models deception using signaling games. But signaling games inherently model deception that is undetectable. In this paper, we extend signaling games by including a detector that gives off probabilistic warnings when the sender acts deceptively. Then we derive pooling and partially-separating equilibria of the game. We find that 1) high-quality detectors eliminate some pure-strategy equilibria, 2) detectors with high true-positive rates encourage more honest signaling than detectors with low false-positive rates, 3) receivers obtain optimal outcomes for equal-error-rate detectors, and 4) surprisingly, deceptive senders sometimes benefit from highly accurate deception detectors. We illustrate these results with an application to defensive deception for network security. Our results provide a quantitative and rigorous analysis of the fundamental aspects of detectable deception.

Jeffrey Pawlick, Edward Colbert, Quanyan Zhu

Cyberattacks on both databases and critical infrastructure have threatened public and private sectors. Ubiquitous tracking and wearable computing have infringed upon privacy. Advocates and engineers have recently proposed using defensive deception as a means to leverage the information asymmetry typically enjoyed by attackers as a tool for defenders. The term deception, however, has been employed broadly and with a variety of meanings. In this paper, we survey 24 articles from 2008-2018 that use game theory to model defensive deception for cybersecurity and privacy. Then we propose a taxonomy that defines six types of deception: perturbation, moving target defense, obfuscation, mixing, honey-x, and attacker engagement. These types are delineated by their information structures, agents, actions, and duration: precisely concepts captured by game theory. Our aims are to rigorously define types of defensive deception, to capture a snapshot of the state of the literature, to provide a menu of models which can be used for applied research, and to identify promising areas for future work. Our taxonomy provides a systematic foundation for understanding different types of defensive deception commonly encountered in cybersecurity and privacy.

Jeffrey Pawlick, Quanyan Zhu

Deception plays a critical role in many interactions in communication and network security. Game-theoretic models called "cheap talk signaling games" capture the dynamic and information asymmetric nature of deceptive interactions. But signaling games inherently model undetectable deception. In this paper, we investigate a model of signaling games in which the receiver can detect deception with some probability. This model nests traditional signaling games and complete information Stackelberg games as special cases. We present the pure strategy perfect Bayesian Nash equilibria of the game. Then we illustrate these analytical results with an application to active network defense. The presence of evidence forces majority-truthful behavior and eliminates some pure strategy equilibria. It always benefits the deceived player, but surprisingly sometimes also benefits the deceiving player.

Jeffrey Pawlick, Thi Thu Hang Nguyen, Edward Colbert et al.

Advanced persistent threats (APTs) are stealthy attacks which make use of social engineering and deception to give adversaries insider access to networked systems. Against APTs, active defense technologies aim to create and exploit information asymmetry for defenders. In this paper, we study a scenario in which a powerful defender uses honeynets for active defense in order to observe an attacker who has penetrated the network. Rather than immediately eject the attacker, the defender may elect to gather information. We introduce an undiscounted, infinite-horizon Markov decision process on a continuous state space in order to model the defender's problem. We find a threshold of information that the defender should gather about the attacker before ejecting him. Then we study the robustness of this policy using a Stackelberg game. Finally, we simulate the policy for a conceptual network. Our results provide a quantitative foundation for studying optimal timing for attacker engagement in network defense.

Jeffrey Pawlick, Quanyan Zhu

While the Internet of things (IoT) promises to improve areas such as energy efficiency, health care, and transportation, it is highly vulnerable to cyberattacks. In particular, distributed denial-of-service (DDoS) attacks overload the bandwidth of a server. But many IoT devices form part of cyber-physical systems (CPS). Therefore, they can be used to launch "physical" denial-of-service attacks (PDoS) in which IoT devices overflow the "physical bandwidth" of a CPS. In this paper, we quantify the population-based risk to a group of IoT devices targeted by malware for a PDoS attack. In order to model the recruitment of bots, we develop a "Poisson signaling game," a signaling game with an unknown number of receivers, which have varying abilities to detect deception. Then we use a version of this game to analyze two mechanisms (legal and economic) to deter botnet recruitment. Equilibrium results indicate that 1) defenders can bound botnet activity, and 2) legislating a minimum level of security has only a limited effect, while incentivizing active defense can decrease botnet activity arbitrarily. This work provides a quantitative foundation for proactive PDoS defense.

Jeffrey Pawlick, Quanyan Zhu

Data ecosystems are becoming larger and more complex due to online tracking, wearable computing, and the Internet of Things. But privacy concerns are threatening to erode the potential benefits of these systems. Recently, users have developed obfuscation techniques that issue fake search engine queries, undermine location tracking algorithms, or evade government surveillance. Interestingly, these techniques raise two conflicts: one between each user and the machine learning algorithms which track the users, and one between the users themselves. In this paper, we use game theory to capture the first conflict with a Stackelberg game and the second conflict with a mean field game. We combine both into a dynamic and strategic bi-level framework which quantifies accuracy using empirical risk minimization and privacy using differential privacy. In equilibrium, we identify necessary and sufficient conditions under which 1) each user is incentivized to obfuscate if other users are obfuscating, 2) the tracking algorithm can avoid this by promising a level of privacy protection, and 3) this promise is incentive-compatible for the tracking algorithm.

Jeffrey Pawlick, Quanyan Zhu

While the Internet of things (IoT) promises to improve areas such as energy efficiency, health care, and transportation, it is highly vulnerable to cyberattacks. In particular, DDoS attacks work by overflowing the bandwidth of a server. But many IoT devices form part of cyber-physical systems (CPS). Therefore, they can be used to launch a "physical" denial-of-service attack (PDoS) in which IoT devices overflow the "physical bandwidth" of a CPS. In this paper, we quantify the population-based risk to a group of IoT devices targeted by malware for a PDoS attack. To model the recruitment of bots, we extend a traditional game-theoretic concept and create a "Poisson signaling game." Then we analyze two different mechanisms (legal and economic) to deter botnet recruitment. We find that 1) defenders can bound botnet activity and 2) legislating a minimum level of security has only a limited effect, while incentivizing active defense can decrease botnet activity arbitrarily. This work provides a quantitative foundation for designing proactive defense against PDoS attacks.

Jeffrey Pawlick, Quanyan Zhu

Strategic interactions ranging from politics and pharmaceuticals to e-commerce and social networks support equilibria in which agents with private information manipulate others which are vulnerable to deception. Especially in cyberspace and the Internet of things, deception is difficult to detect and trust is complicated to establish. For this reason, effective policy-making, profitable entrepreneurship, and optimal technological design demand quantitative models of deception. In this paper, we use game theory to model specifically one-to-many deception. We combine a signaling game with a model called a Poisson game. The resulting Poisson signaling game extends traditional signaling games to include 1) exogenous evidence of deception, 2) an unknown number of receivers, and 3) receivers of multiple types. We find closed-form equilibrium solutions for a subset of Poisson signaling games, and characterize the rates of deception that they support. We show that receivers with higher abilities to detect deception can use crowd-defense tactics to mitigate deception for receivers with lower abilities to detect deception. Finally, we discuss how Poisson signaling games could be used to defend against the process by which the Mirai botnet recruits IoT devices in preparation for a distributed denial-of-service attack.

Jeffrey Pawlick, Quanyan Zhu

Data is the new oil; this refrain is repeated extensively in the age of internet tracking, machine learning, and data analytics. As data collection becomes more personal and pervasive, however, public pressure is mounting for privacy protection. In this atmosphere, developers have created applications to add noise to user attributes visible to tracking algorithms. This creates a strategic interaction between trackers and users when incentives to maintain privacy and improve accuracy are misaligned. In this paper, we conceptualize this conflict through an N+1-player, augmented Stackelberg game. First a machine learner declares a privacy protection level, and then users respond by choosing their own perturbation amounts. We use the general frameworks of differential privacy and empirical risk minimization to quantify the utility components due to privacy and accuracy, respectively. In equilibrium, each user perturbs her data independently, which leads to a high net loss in accuracy. To remedy this scenario, we show that the learner improves his utility by proactively perturbing the data himself. While other work in this area has studied privacy markets and mechanism design for truthful reporting of user information, we take a different viewpoint by considering both user and learner perturbation.

Jeffrey Pawlick, Quanyan Zhu

Data is the new oil; this refrain is repeated extensively in the age of internet tracking, machine learning, and data analytics. Social network analysis, cookie-based advertising, and government surveillance are all evidence of the use of data for commercial and national interests. Public pressure, however, is mounting for the protection of privacy. Frameworks such as differential privacy offer machine learning algorithms methods to guarantee limits to information disclosure, but they are seldom implemented. Recently, however, developers have made significant efforts to undermine tracking through obfuscation tools that hide user characteristics in a sea of noise. These services highlight an emerging clash between tracking and data obfuscation. In this paper, we conceptualize this conflict through a dynamic game between users and a machine learning algorithm that uses empirical risk minimization. First, a machine learner declares a privacy protection level, and then users respond by choosing their own perturbation amounts. We study the interaction between the users and the learner using a Stackelberg game. The utility functions quantify accuracy using expected loss and privacy in terms of the bounds of differential privacy. In equilibrium, we find selfish users tend to cause significant utility loss to trackers by perturbing heavily, in a phenomenon reminiscent of public good games. Trackers, however, can improve the balance by proactively perturbing the data themselves. While other work in this area has studied privacy markets and mechanism design for truthful reporting of user information, we take a different viewpoint by considering both user and learner perturbation.

Jeffrey Pawlick, Quanyan Zhu

Internet tracking technologies and wearable electronics provide a vast amount of data to machine learning algorithms. This stock of data stands to increase with the developments of the internet of things and cyber-physical systems. Clearly, these technologies promise benefits. But they also raise the risk of sensitive information disclosure. To mitigate this risk, machine learning algorithms can add noise to outputs according to the formulations provided by differential privacy. At the same time, users can fight for privacy by injecting noise into the data that they report. In this paper, we conceptualize the interactions between privacy and accuracy and between user (input) perturbation and learner (output) perturbation in machine learning, using the frameworks of empirical risk minimization, differential privacy, and Stackelberg games. In particular, we solve for the Stackelberg equilibrium for the case of an averaging query. We find that, in equilibrium, either the users perturb their data before submission or the learner perturbs the machine learning output, but never both. Specifically, the learner perturbs if and only if the number of users is greater than a threshold which increases with the degree to which incentives are misaligned. Provoked by these conclusions - and by some observations from privacy ethics - we also suggest future directions. While other work in this area has studied privacy markets and mechanism design for truthful reporting of user information, we take a different viewpoint by considering both user and learner perturbation. We hope that this effort will open the door to future work in the area of differential privacy games.

Jeffrey Pawlick, Sadegh Farhang, Quanyan Zhu

Access to the cloud has the potential to provide scalable and cost effective enhancements of physical devices through the use of advanced computational processes run on apparently limitless cyber infrastructure. On the other hand, cyber-physical systems and cloud-controlled devices are subject to numerous design challenges; among them is that of security. In particular, recent advances in adversary technology pose Advanced Persistent Threats (APTs) which may stealthily and completely compromise a cyber system. In this paper, we design a framework for the security of cloud-based systems that specifies when a device should trust commands from the cloud which may be compromised. This interaction can be considered as a game between three players: a cloud defender/administrator, an attacker, and a device. We use traditional signaling games to model the interaction between the cloud and the device, and we use the recently proposed FlipIt game to model the struggle between the defender and attacker for control of the cloud. Because attacks upon the cloud can occur without knowledge of the defender, we assume that strategies in both games are picked according to prior commitment. This framework requires a new equilibrium concept, which we call Gestalt Equilibrium, a fixed-point that expresses the interdependence of the signaling and FlipIt games. We present the solution to this fixed-point problem under certain parameter cases, and illustrate an example application of cloud control of an unmanned vehicle. Our results contribute to the growing understanding of cloud-controlled systems.

Saboor Zahir, John Pak, Jatinder Singh et al.

Cyber literacy merits serious research attention because it addresses a confluence of specialization and generalization; cybersecurity is often conceived of as approachable only by a technological intelligentsia, yet its interdependent nature demands education for a broad population. Therefore, educational tools should lead participants to discover technical knowledge in an accessible and attractive framework. In this paper, we present Protection and Deception (P&G), a novel two-player board game. P&G has three main contributions. First, it builds cyber literacy by giving participants "hands-on" experience with game pieces that have the capabilities of cyber-attacks such as worms, masquerading attacks/spoofs, replay attacks, and Trojans. Second, P&G teaches the important game-theoretic concepts of asymmetric information and resource allocation implicitly and non-obtrusively through its game play. Finally, it strives for the important objective of security education for underrepresented minorities and people without explicit technical experience. We tested P&G at a community center in Manhattan with middle- and high school students, and observed enjoyment and increased cyber literacy along with suggestions for improvement of the game. Together with these results, our paper also presents images of the attractive board design and 3D printed game pieces, together with a Monte-Carlo analysis that we used to ensure a balanced gaming experience.

Jeffrey Pawlick, Quanyan Zhu

Deception plays a critical role in the financial industry, online markets, national defense, and countless other areas. Understanding and harnessing deception - especially in cyberspace - is both crucial and difficult. Recent work in this area has used game theory to study the roles of incentives and rational behavior. Building upon this work, we employ a game-theoretic model for the purpose of mechanism design. Specifically, we study a defensive use of deception: implementation of honeypots for network defense. How does the design problem change when an adversary develops the ability to detect honeypots? We analyze two models: cheap-talk games and an augmented version of those games that we call cheap-talk games with evidence, in which the receiver can detect deception with some probability. Our first contribution is this new model for deceptive interactions. We show that the model includes traditional signaling games and complete information games as special cases. We also demonstrate numerically that deception detection sometimes eliminate pure-strategy equilibria. Finally, we present the surprising result that the utility of a deceptive defender can sometimes increase when an adversary develops the ability to detect deception. These results apply concretely to network defense. They are also general enough for the large and critical body of strategic interactions that involve deception.