Abhinav Palia

Alexandra Chouldechova, A. Feder Cooper, Solon Barocas et al.

We argue that conclusions drawn about relative system safety or attack method efficacy via AI red teaming are often not supported by evidence provided by attack success rate (ASR) comparisons. We show, through conceptual, theoretical, and empirical contributions, that many conclusions are founded on apples-to-oranges comparisons or low-validity measurements. Our arguments are grounded in asking a simple question: When can attack success rates be meaningfully compared? To answer this question, we draw on ideas from social science measurement theory and inferential statistics, which, taken together, provide a conceptual grounding for understanding when numerical values obtained through the quantification of system attributes can be meaningfully compared. Through this lens, we articulate conditions under which ASRs can and cannot be meaningfully compared. Using jailbreaking as a running example, we provide examples and extensive discussion of apples-to-oranges ASR comparisons and measurement validity challenges.

Abhinav Palia, Rajat Tandon, Carl Mathis

Privacy is an individual choice to determine which personal details can be collected, used and shared. Individual consent and transparency are the core tenets for earning customers trust and this motivates the organizations to adopt privacy enhancing practices while creating the systems. The goal of a privacy-aware design is to protect information in a way that does not increase an adversary's existing knowledge about an individual beyond what is permissible. This becomes critical when these data elements can be linked with the wealth of auxiliary information available outside the system to identify an individual. Privacy regulations around the world provide directives to protect individual privacy but are generally complex and vague, making their translation into actionable and technical privacy-friendly architectures challenging. In this paper, we utilize Shannon's Entropy to create an objective metric that can help simplify the state-of-the-art Privacy Design Strategies proposed in the literature and aid our key technical design decisions to create privacy aware architectures.

Abhinav Palia, Caroline Devlin, Megan Yelorda

Any program that is designed to accomplish certain objectives, needs to establish program level controls pertaining to the overall goal. A critical aspect that determines the success of a program is the quality of the controls and their effectiveness in accomplishing the goal. Traditional Control Maturity Models primarily focus on the efficiency, management, and optimization of controls, while only indirectly measuring control effectiveness which neglects an essential aspect of control efficacy. In this paper, we highlight the ineffectiveness of these models, outline an adaptable program controls framework, and provide an approach to define measurable attributes (metrics) that enable a zero-defect program. To the best of our knowledge, we believe this is the first paper that provides a structured approach to defining a controls measurement framework and creation of effectiveness metrics that can be adopted by a variety of program use cases.

Baiyu Liu, Abhinav Palia, Shan-Ho Yang

Along with the classical problem of managing multiple identities, actions, devices, APIs etc. in different businesses, there has been an escalating need for having the capability of flexible attribute based access control~(ABAC) mechanisms. In order to fill this gap, several variations of ABAC model have been proposed such as \textit{Amazon's AWS IAM}, which uses JSON as their underlying storage data structure and adds policies/constraints as fields over the regular ABAC. However, these systems still do not provide the capability to have customized permissions and to perform various operations (such as comparison/aggregation) on them. In this paper, we introduce a string based resource naming strategy that supports the customized and conditional permissions for resource access. Further, we propose the basic architecture of our system which, along with our naming scheme, makes the system scalable, secure, efficient, flexible and customizable. Finally, we present the proof of concept for our algorithm as well as the experimental set up and the future trajectory for this work.

Abhinav Palia, Rajat Tandon

With the tremendous increase in the number of smart phones, app stores have been overwhelmed with applications requiring geo-location access in order to provide their users better services through personalization. Revealing a user's location to these third party apps, no matter at what frequency, is a severe privacy breach which can have unpleasant social consequences. In order to prevent inference attacks derived from geo-location data, a number of location obfuscation techniques have been proposed in the literature. However, none of them provides any objective measure of privacy guarantee. Some work has been done to define differential privacy for geo-location data in the form of geo-indistinguishability with l privacy guarantee. These techniques do not utilize any prior background information about the Points of Interest (PoIs) of a user and apply Laplacian noise to perturb all the location coordinates. Intuitively, the utility of such a mechanism can be improved if the noise distribution is derived after considering some prior information about PoIs. In this paper, we apply the standard definition of differential privacy on geo-location data. We use first principles to model various privacy and utility constraints, prior background information available about the PoIs (distribution of PoI locations in a 1D plane) and the granularity of the input required by different types of apps, in order to produce a more accurate and a utility maximizing differentially private algorithm for geo-location data at the OS level. We investigate this for a particular category of apps and for some specific scenarios. This will also help us to verify that whether Laplacian noise is still the optimal perturbation when we have such prior information.