Parastoo Sadeghi

Ni Ding, Parastoo Sadeghi, Rodney A. Kennedy

We study the adaptive modulation (AM) problem in a network-coded two-way relay channel (NC-TWRC), where each of the two users controls its own bit rate in the $m$-ary quadrature amplitude modulation ($m$-QAM) to minimize the transmission error rate and enhance the spectral efficiency. We show that there exists a strategic complementarity, one user tends to transmit while the other decides to do so in order to enhance the overall spectral efficiency, which is beyond the scope of the conventional single-agent AM scheduling method. We propose a two-player game model parameterized by the signal-to-noise ratios (SNRs) of two user-to-user channels and prove that it is a supermodular game where there always exist the extremal pure strategy Nash equilibria (PSNEs), the largest and smallest PSNEs. We show by simulation results that the extremal PSNEs incur a similar bit error rate (BER) as the conventional single-agent AM scheme, but significantly improve the spectral efficiency in the NC-TWRC system. The study also reveals the Pareto order of the extremal PSNEs: The largest and smallest PSNEs are Pareto worst and best PSNEs, respectively. Finally, we derive the sufficient conditions for the extremal PSNEs to be symmetric and monotonic in channel SNRs. We also discuss how to utilize the symmetry and monotonicity to relieve the complexity in the PSNE learning process.

Sara Saeidian, Leonhard Grosse, Parastoo Sadeghi et al.

This paper explores the implications of guaranteeing privacy by imposing a lower bound on the information density between the private and the public data. We introduce a novel and operationally meaningful privacy measure called pointwise maximal cost (PMC) and demonstrate that imposing an upper bound on PMC is equivalent to enforcing a lower bound on the information density. PMC quantifies the information leakage about a secret to adversaries who aim to minimize non-negative cost functions after observing the outcome of a privacy mechanism. When restricted to finite alphabets, PMC can equivalently be defined as the information leakage to adversaries aiming to minimize the probability of incorrectly guessing randomized functions of the secret. We study the properties of PMC and apply it to standard privacy mechanisms to demonstrate its practical relevance. Through a detailed examination, we connect PMC with other privacy measures that impose upper or lower bounds on the information density. These are pointwise maximal leakage (PML), local differential privacy (LDP), and (asymmetric) local information privacy. In particular, we show that a mechanism satisfies LDP if and only if it has both bounded PMC and bounded PML. Overall, our work fills a conceptual and operational gap in the taxonomy of privacy measures, bridges existing disconnects between different frameworks, and offers insights for selecting a suitable notion of privacy in a given application.

Amirreza Zamani, Sajad Daei, Parastoo Sadeghi et al.

We study sparse locally private channels of the form $M(y\mid x)\propto w(x,y) 1\{y\in S(x)\},$ where the admissible output set $S(x)$ is allowed to depend on the private input $x$ and is assumed to be small. Here, we consider the sparse discrete-Laplace family with kernel $w(x,y)=e^{-λd(x,y)}$ and the sparse Gaussian family with kernel $w(x,y)=e^{-d(x,y)^2/(2σ^2)}$. For both families we give exact characterizations of pure and approximate local differential privacy. For pure $\varepsilon$-local differential privacy, we show that input-dependent sparse supports are obtained when all supports coincide. For $(\varepsilon,δ)$-local differential privacy, we derive exact formulas for the privacy defect in terms of support leakage and excess privacy loss on the overlap region. We then specialize the analysis to radius-truncated sparse discrete-Laplace and radius-truncated sparse Gaussian mechanisms and obtain explicit privacy-sparsity tradeoffs in terms of the support size $s$. In particular, we show that nontrivial approximate local privacy requires a minimum support size, whereas larger supports reduce support leakage but increase distortion. For the Gaussian family, the overlap term exhibits an additional quadratic dependence on the support radius, which implies a sharper tradeoff between privacy and sparsity. These results identify the support cardinality as the intrinsic complexity parameter of the mechanism and yield an optimal design principle: choose the smallest support size that satisfies the target privacy constraint.

Sayan Biswas, Mark Dras, Pedro Faustini et al.

Within the machine learning community, reconstruction attacks are a principal concern and have been identified even in federated learning (FL), which was designed with privacy preservation in mind. In response to these threats, the privacy community recommends the use of differential privacy (DP) in the stochastic gradient descent algorithm, termed DP-SGD. However, the proliferation of variants of DP in recent years\textemdash such as metric privacy\textemdash has made it challenging to conduct a fair comparison between different mechanisms due to the different meanings of the privacy parameters $ε$ and $δ$ across different variants. Thus, interpreting the practical implications of $ε$ and $δ$ in the FL context and amongst variants of DP remains ambiguous. In this paper, we lay a foundational framework for comparing mechanisms with differing notions of privacy guarantees, namely $(ε,δ)$-DP and metric privacy. We provide two foundational means of comparison: firstly, via the well-established $(ε,δ)$-DP guarantees, made possible through the Rényi differential privacy framework; and secondly, via Bayes' capacity, which we identify as an appropriate measure for reconstruction threats.

Sayan Biswas, Mark Dras, Pedro Faustini et al.

Within the machine learning community, reconstruction attacks are a principal attack of concern and have been identified even in federated learning, which was designed with privacy preservation in mind. In federated learning, it has been shown that an adversary with knowledge of the machine learning architecture is able to infer the exact value of a training element given an observation of the weight updates performed during stochastic gradient descent. In response to these threats, the privacy community recommends the use of differential privacy in the stochastic gradient descent algorithm, termed DP-SGD. However, DP has not yet been formally established as an effective countermeasure against reconstruction attacks. In this paper, we formalise the reconstruction threat model using the information-theoretic framework of quantitative information flow. We show that the Bayes' capacity, related to the Sibson mutual information of order infinity, represents a tight upper bound on the leakage of the DP-SGD algorithm to an adversary interested in performing a reconstruction attack. We provide empirical results demonstrating the effectiveness of this measure for comparing mechanisms against reconstruction threats.

Ziqi Zhou, Onur Günlü, Rafael G. L. D'Oliveira et al.

We extend a previous framework for designing differentially private (DP) mechanisms via randomized graph colorings that was restricted to binary functions, corresponding to colorings in a graph, to multi-valued functions. As before, datasets are nodes in the graph and any two neighboring datasets are connected by an edge. In our setting, we assume that each dataset has a preferential ordering for the possible outputs of the mechanism, each of which we refer to as a rainbow. Different rainbows partition the graph of datasets into different regions. We show that if the DP mechanism is pre-specified at the boundary of such regions and behaves identically for all same-rainbow boundary datasets, at most one optimal such mechanism can exist and the problem can be solved by means of a morphism to a line graph. We then show closed form expressions for the line graph in the case of ternary functions. Treatment of ternary queries in this paper displays enough richness to be extended to higher-dimensional query spaces with preferential query ordering, but the optimality proof does not seem to follow directly from the ternary proof.

Parastoo Sadeghi, Mehdi Korki

The Gaussian distribution is widely used in mechanism design for differential privacy (DP). Thanks to its sub-Gaussian tail, it significantly reduces the chance of outliers when responding to queries. However, it can only provide approximate $(ε, δ(ε))$-DP. In practice, $δ(ε)$ must be much smaller than the size of the dataset, which may limit the use of the Gaussian mechanism for large datasets with strong privacy requirements. In this paper, we introduce and analyze a new distribution for use in DP that is based on the Gaussian distribution, but has improved privacy performance. The so-called offset-symmetric Gaussian tail (OSGT) distribution is obtained through using the normalized tails of two symmetric Gaussians around zero. Consequently, it can still have sub-Gaussian tail and lend itself to analytical derivations. We analytically derive the variance of the OSGT random variable and the $δ(ε)$ of the OSGT mechanism. We then numerically show that at the same variance, the OSGT mechanism can offer a lower $δ(ε)$ than the Gaussian mechanism. We extend the OSGT mechanism to $k$-dimensional queries and derive an easy-to-compute analytical upper bound for its zero-concentrated differential privacy (zCDP) performance. We analytically prove that at the same variance, the same global query sensitivity and for sufficiently large concentration orders $α$, the OSGT mechanism performs better than the Gaussian mechanism in terms of zCDP.

Yucheng Liu, Lawrence Ong, Sarah Johnson et al.

We study the information leakage to a guessing adversary in zero-error source coding. The source coding problem is defined by a confusion graph capturing the distinguishability between source symbols. The information leakage is measured by the ratio of the adversary's successful guessing probability after and before eavesdropping the codeword, maximized over all possible source distributions. Such measurement under the basic adversarial model where the adversary makes a single guess and allows no distortion between its estimator and the true sequence is known as the maximum min-entropy leakage or the maximal leakage in the literature. We develop a single-letter characterization of the optimal normalized leakage under the basic adversarial model, together with an optimum-achieving scalar stochastic mapping scheme. An interesting observation is that the optimal normalized leakage is equal to the optimal compression rate with fixed-length source codes, both of which can be simultaneously achieved by some deterministic coding schemes. We then extend the leakage measurement to generalized adversarial models where the adversary makes multiple guesses and allows certain level of distortion, for which we derive single-letter lower and upper bounds.

Rafael G. L. D'Oliveira, Salman Salamatian, Muriel Médard et al.

We study the relationship between randomized low influence functions and differentially private mechanisms. Our main aim is to formally determine whether differentially private mechanisms are low influence and whether low influence randomized functions can be differentially private. We show that differential privacy does not necessarily imply low influence in a formal sense. However, low influence implies approximate differential privacy. These results hold for both independent and non-independent randomized mechanisms, where an important instance of the former is the widely-used additive noise techniques in the differential privacy literature. Our study also reveals the interesting dynamics between utility, low influence, and independence of a differentially private mechanism. As the name of this paper suggests, we show that any two such features are simultaneously possible. However, in order to have a differentially private mechanism that has both utility and low influence, even under a very mild utility condition, one has to employ non-independent mechanisms.

Ni Ding, Parastoo Sadeghi, Rodney A. Kennedy

This paper considers a cross-layer adaptive modulation system that is modeled as a Markov decision process (MDP). We study how to utilize the monotonicity of the optimal transmission policy to relieve the computational complexity of dynamic programming (DP). In this system, a scheduler controls the bit rate of the m-quadrature amplitude modulation (m-QAM) in order to minimize the long-term losses incurred by the queue overflow in the data link layer and the transmission power consumption in the physical layer. The work is done in two steps. Firstly, we observe the L-natural-convexity and submodularity of DP to prove that the optimal policy is always nondecreasing in queue occupancy/state and derive the sufficient condition for it to be nondecreasing in both queue and channel states. We also show that, due to the L-natural-convexity of DP, the variation of the optimal policy in queue state is restricted by a bounded marginal effect: The increment of the optimal policy between adjacent queue states is no greater than one. Secondly, we use the monotonicity results to present two low complexity algorithms: monotonic policy iteration (MPI) based on L-natural-convexity and discrete simultaneous perturbation stochastic approximation (DSPSA). We run experiments to show that the time complexity of MPI based on L-natural-convexity is much lower than that of DP and the conventional MPI that is based on submodularity and DSPSA is able to adaptively track the optimal policy when the system parameters change.

Ni Ding, Parastoo Sadeghi, Rodney A. Kennedy

This paper considers the discrete convexity of a cross-layer on-off transmission control problem in wireless communications. In this system, a scheduler decides whether or not to transmit in order to optimize the long-term quality of service (QoS) incurred by the queueing effects in the data link layer and the transmission power consumption in the physical (PHY) layer simultaneously. Using a Markov decision process (MDP) formulation, we show that the optimal policy can be determined by solving a minimization problem over a set of queue thresholds if the dynamic programming (DP) is submodular. We prove that this minimization problem is discrete convex. In order to search the minimizer, we consider two discrete stochastic approximation (DSA) algorithms: discrete simultaneous perturbation stochastic approximation (DSPSA) and L-natural-convex stochastic approximation (L-natural-convex SA). Through numerical studies, we show that the two DSA algorithms converge significantly faster than the existing continuous simultaneous perturbation stochastic approximation (CSPSA) algorithm in multi-user systems. Finally, we compare the convergence results and complexity of two DSA and CSPSA algorithms where we show that DSPSA achieves the best trade-off between complexity and accuracy in multi-user systems.

Ni Ding, Parastoo Sadeghi, Rodney A. Kennedy

This paper considers a transmission control problem in network-coded two-way relay channels (NC-TWRC), where the relay buffers random symbol arrivals from two users, and the channels are assumed to be fading. The problem is modeled by a discounted infinite horizon Markov decision process (MDP). The objective is to find a transmission control policy that minimizes the symbol delay, buffer overflow and transmission power consumption and error rate simultaneously and in the long run. By using the concepts of submodularity, multimodularity and L-natural convexity, we study the structure of the optimal policy searched by dynamic programming (DP) algorithm. We show that the optimal transmission policy is nondecreasing in queue occupancies or/and channel states under certain conditions such as the chosen values of parameters in the MDP model, channel modeling method, modulation scheme and the preservation of stochastic dominance in the transitions of system states. The results derived in this paper can be used to relieve the high complexity of DP and facilitate real-time control.

Shahriar Etemadi Tajbakhsh, Parastoo Sadeghi, Rodney Kennedy

We introduce a method for securely delivering a set of messages to a group of clients over a broadcast erasure channel where each client is interested in a distinct message. Each client is able to obtain its own message but not the others'. In the proposed method the messages are combined together using a special variant of random linear network coding. Each client is provided with a private set of decoding coefficients to decode its own message. Our method provides security for the transmission sessions against computational brute-force attacks and also weakly security in information theoretic sense. As the broadcast channel is assumed to be erroneous, the missing coded packets should be recovered in some way. We consider two different scenarios. In the first scenario the missing packets are retransmitted by the base station (centralized). In the second scenario the clients cooperate with each other by exchanging packets (decentralized). In both scenarios, network coding techniques are exploited to increase the total throughput. For the case of centralized retransmissions we provide an analytical approximation for the throughput performance of instantly decodable network coded (IDNC) retransmissions as well as numerical experiments. For the decentralized scenario, we propose a new IDNC based retransmission method where its performance is evaluated via simulations and analytical approximation. Application of this method is not limited to our special problem and can be generalized to a new class of problems introduced in this paper as the cooperative index coding problem.

Shahriar Etemadi Tajbakhsh, Parastoo Sadeghi

We consider a set of $n$ messages and a group of $k$ clients. Each client is privileged for receiving an arbitrary subset of the messages over a broadcast erasure channel, which generalizes scenario of a previous work. We propose a method for secretly delivering each message to its privileged recipients in a way that each receiver can decode its own messages but not the others'. Our method is based on combining the messages using linear network coding and hiding the decoding coefficients from the unprivileged clients. We provide an information theoretic proof for the secrecy of the proposed method. In particular we show that an unprivileged client cannot obtain any meaningful information even if it holds the entire set of coded data packets transmitted over the channel. Moreover, in our method, the decoding complexity is desirably low at the receiver side.