Aaron B. Wagner

Sourbh Bhadane, Aaron B. Wagner, Johannes Ballé

Artificial Neural-Network-based (ANN-based) lossy compressors have recently obtained striking results on several sources. Their success may be ascribed to an ability to identify the structure of low-dimensional manifolds in high-dimensional ambient spaces. Indeed, prior work has shown that ANN-based compressors can achieve the optimal entropy-distortion curve for some such sources. In contrast, we determine the optimal entropy-distortion tradeoffs for two low-dimensional manifolds with circular structure and show that state-of-the-art ANN-based compressors fail to optimally compress them.

Yang Qiu, Aaron B. Wagner, Johannes Ballé et al.

We introduce a distortion measure for images, Wasserstein distortion, that simultaneously generalizes pixel-level fidelity on the one hand and realism or perceptual quality on the other. We show how Wasserstein distortion reduces to a pure fidelity constraint or a pure realism constraint under different parameter choices and discuss its metric properties. Pairs of images that are close under Wasserstein distortion illustrate its utility. In particular, we generate random textures that have high fidelity to a reference texture in one location of the image and smoothly transition to an independent realization of the texture as one moves away from this point. Wasserstein distortion attempts to generalize and unify prior work on texture generation, image realism and distortion, and models of the early human visual system, in the form of an optimizable metric in the mathematical sense.

Adeel Mahmood, Aaron B. Wagner

Through refined asymptotic analysis based on the normal approximation, we study how higher-order coding performance depends on the mean power $Γ$ as well as on finer statistics of the input power. We introduce a multifaceted power model in which the expectation of an arbitrary (but finite) number of arbitrary functions of the normalized average power is constrained. The framework generalizes existing models, recovering the standard maximal and expected power constraints and the recent mean and variance constraint as special cases. Under certain growth and continuity assumptions on the functions, our main theorem gives an exact characterization of the minimum average error probability for Gaussian channels as a function of the first- and second-order coding rates. The converse proof reduces the code design problem to minimization over a compact (under the Prokhorov metric) set of probability distributions, characterizes the extreme points of this set and invokes the Bauer's maximization principle. Our results for the multifaceted power model serve as more precise benchmarks for practical modulation schemes with multiple amplitude levels, probabilistic shaping and nonuniform constellation geometries.

Sourbh Bhadane, Aaron B. Wagner

We consider the one-bit quantizer that minimizes the mean squared error for a source living in a real Hilbert space. The optimal quantizer is a projection followed by a thresholding operation, and we provide methods for identifying the optimal direction along which to project. As an application of our methods, we characterize the optimal one-bit quantizer for a continuous-time random process that exhibits low-dimensional structure. We numerically show that this optimal quantizer is found by a neural-network-based compressor trained via stochastic gradient descent.

Aaron B. Wagner

A rate-distortion-perception (RDP) tradeoff has recently been proposed by Blau and Michaeli and also Matsumoto. Focusing on the case of perfect realism, which coincides with the problem of distribution-preserving lossy compression studied by Li et al., a coding theorem for the RDP tradeoff that allows for a specified amount of common randomness between the encoder and decoder is provided. The existing RDP tradeoff is recovered by allowing for the amount of common randomness to be infinite. The quadratic Gaussian case is examined in detail.

Sourbh Bhadane, Aaron B. Wagner, Jayadev Acharya

We consider a linear autoencoder in which the latent variables are quantized, or corrupted by noise, and the constraint is Schur-concave in the set of latent variances. Although finding the optimal encoder/decoder pair for this setup is a nonconvex optimization problem, we show that decomposing the source into its principal components is optimal. If the constraint is strictly Schur-concave and the empirical covariance matrix has only simple eigenvalues, then any optimal encoder/decoder must decompose the source in this way. As one application, we consider a strictly Schur-concave constraint that estimates the number of bits needed to represent the latent variables under fixed-rate encoding, a setup that we call \emph{Principal Bit Analysis (PBA)}. This yields a practical, general-purpose, fixed-rate compressor that outperforms existing algorithms. As a second application, we show that a prototypical autoencoder-based variable-rate compressor is guaranteed to decompose the source into its principal components.

Lucas Theis, Aaron B. Wagner

The rate-distortion-perception function (RDPF; Blau and Michaeli, 2019) has emerged as a useful tool for thinking about realism and distortion of reconstructions in lossy compression. Unlike the rate-distortion function, however, it is unknown whether encoders and decoders exist that achieve the rate suggested by the RDPF. Building on results by Li and El Gamal (2018), we show that the RDPF can indeed be achieved using stochastic, variable-length codes. For this class of codes, we also prove that the RDPF lower-bounds the achievable rate

Benjamin Wu, Aaron B. Wagner, G. Edward Suh

Side channels represent a broad class of security vulnerabilities that have been demonstrated to exist in many applications. Because completely eliminating side channels often leads to prohibitively high overhead, there is a need for a principled trade-off between cost and leakage. In this paper, we make a case for the use of maximal leakage to analyze such trade-offs. Maximal leakage is an operationally interpretable leakage metric designed for side channels. We present the most useful theoretical properties of maximal leakage from previous work and demonstrate empirically that conventional metrics such as mutual information and channel capacity underestimate the threat posed by side channels whereas maximal leakage does not. We also study the cost-leakage trade-off as an optimization problem using maximal leakage. We demonstrate that not only can this problem be represented as a linear program, but also that optimal protection can be achieved using a combination of at most two deterministic schemes.