Matthieu R. Bloch

Tuna Erdoğan, Shi-Yuan Wang, Shang-Jen Su et al.

We consider a joint communication and sensing problem over an optical link in which a low-power transmitter simultaneously communicates with a receiver and identifies the range of a defect producing a backscattered signal. We model the system as a lossy thermal-noise bosonic channel, in which the target location, modeled as a beamsplitter, affects the timing of the backscattered signal. Motivated by the envisioned deployment of entanglement-enabled quantum networks, we allow the transmitter to exploit shared entanglement to assist both sensing and communication. Since entanglement is known to enhance sensing, as demonstrated in Quantum Illumination (QI), and to increase communication rates through entanglement-assisted communication, the transmitter faces a trade-off in allocating its entanglement resources between the two tasks. Our main result is a characterization of these trade-offs in the form of an achievable rate/error-exponent region, which can outperform time-sharing and demonstrates a quantum advantage.

Sidong Guo, Tyler Kann, Teodora Baluta et al.

We study the problem of multi-bit watermarking for large language models (LLMs). We introduce a block-autoregressive model inspired by multi-token prediction, in which the encoder has limited non-causal access to token distributions within each block. This formulation enables an information-theoretic characterization of multi-bit watermarking capacity, by which the knowledge of LLM cover statistics is leveraged to enable a multi-bit covert embedding. We study the information-theoretic limits of the model by combining Gelfand-Pinsker and channel synthesis coding techniques and obtain an exact characterization of the capacity. The embedding strategy is further optimized across blocks using a constrained Markov decision process (CMDP) and we develop an explicit algorithm based on polar codes following the information-theoretic principles. Our algorithm achieves a bit-error rate below 10 percent with a rate of 0.375 bits/token over short token lengths with negligible perplexity and distortion degradation.

Belén Martín-Urcelay, Yoonsang Lee, Matthieu R. Bloch et al.

Integrating human expertise into machine learning systems often reduces the role of experts to labeling oracles, a paradigm that limits the amount of information exchanged and fails to capture the nuances of human judgment. We address this challenge by developing a human-in-the-loop framework to learn binary classifiers with rich query types, consisting of item ranking and exemplar selection. We first introduce probabilistic human response models for these rich queries motivated by the relationship experimentally observed between the perceived implicit score of an item and its distance to the unknown classifier. Using these models, we then design active learning algorithms that leverage the rich queries to increase the information gained per interaction. We provide theoretical bounds on sample complexity and develop a tractable and computationally efficient variational approximation. Through experiments with simulated annotators derived from crowdsourced word-sentiment and image-aesthetic datasets, we demonstrate significant reductions on sample complexity. We further extend active learning strategies to select queries that maximize information rate, explicitly balancing informational value against annotation cost. This algorithm in the word sentiment classification task reduces learning time by more than 57\% compared to traditional label-only active learning.

Onur Günlü, Matthieu Bloch, Rafael F. Schaefer et al.

This work considers the problem of mitigating information leakage between communication and sensing in systems jointly performing both operations. Specifically, a discrete memoryless state-dependent broadcast channel model is studied in which (i) the presence of feedback enables a transmitter to convey information, while simultaneously performing channel state estimation; (ii) one of the receivers is treated as an eavesdropper whose state should be estimated but which should remain oblivious to part of the transmitted information. The model abstracts the challenges behind security for joint communication and sensing if one views the channel state as a sensitive attribute, e.g., location. For independent and identically distributed states, perfect output feedback, and when part of the transmitted message should be kept secret, a partial characterization of the secrecy-distortion region is developed. The characterization is exact when the broadcast channel is either physically-degraded or reversely-physically-degraded. The partial characterization is also extended to the situation in which the entire transmitted message should be kept secret. The benefits of a joint approach compared to separation-based secure communication and state-sensing methods are illustrated with a binary joint communication and sensing model.

Onur Günlü, Matthieu Bloch, Rafael F. Schaefer

We consider a distributed function computation problem in which parties observing noisy versions of a remote source facilitate the computation of a function of their observations at a fusion center through public communication. The distributed function computation is subject to constraints, including not only reliability and storage but also privacy and secrecy. Specifically, 1) the remote source should remain private from an eavesdropper and the fusion center, measured in terms of the information leaked about the remote source; 2) the function computed should remain secret from the eavesdropper, measured in terms of the information leaked about the arguments of the function, to ensure secrecy regardless of the exact function used. We derive the exact rate regions for lossless and lossy single-function computation and illustrate the lossy single-function computation rate region for an information bottleneck example, in which the optimal auxiliary random variables are characterized for binary-input symmetric-output channels. We extend the approach to lossless and lossy asynchronous multiple-function computations with joint secrecy and privacy constraints, in which case inner and outer bounds for the rate regions differing only in the Markov chain conditions imposed are characterized.

Gregory Canal, Matthieu Bloch, Christopher Rozell

The iterative selection of examples for labeling in active machine learning is conceptually similar to feedback channel coding in information theory: in both tasks, the objective is to seek a minimal sequence of actions to encode information in the presence of noise. While this high-level overlap has been previously noted, there remain open questions on how to best formulate active learning as a communications system to leverage existing analysis and algorithms in feedback coding. In this work, we formally identify and leverage the structural commonalities between the two problems, including the characterization of encoder and noisy channel components, to design a new algorithm. Specifically, we develop an optimal transport-based feedback coding scheme called Approximate Posterior Matching (APM) for the task of active example selection and explore its application to Bayesian logistic regression, a popular model in active learning. We evaluate APM on a variety of datasets and demonstrate learning performance comparable to existing active learning methods, at a reduced computational cost. These results demonstrate the potential of directly deploying concepts from feedback channel coding to design efficient active learning strategies.

Mehrdad Tahmasbi, Matthieu R. Bloch

We show that covert secret key expansion is possible using a public authenticated classical channel and a quantum channel largely under control of an adversary, which we precisely define. We also prove a converse result showing that, under the golden standard of quantum key distribution by which the adversary completely controls the quantum channel, no covert key generation is possible. We propose a protocol based on pulse-position modulation and multi-level coding that allows one to use traditional quantum key distribution (QKD) protocols while ensuring covertness, in the sense that no statistical test by the adversary can detect the presence of communication better than a random guess. When run over a bosonic channel, our protocol can leverage existing discrete-modulated continuous variable protocols. Since existing techniques to bound Eve's information do not directly apply, we develop a new bound that results in positive throughput for a range of channel parameters.

Vincent Y. F. Tan, Matthieu R. Bloch

We consider block codes for degraded wiretap channels in which the legitimate receiver decodes the message with an asymptotic error probability no larger than $\varepsilon$ but the leakage to the eavesdropper vanishes. For discrete memoryless and Gaussian wiretap channels, we show that the maximum rate of transmission does not depend on $\varepsilon\in [0,1)$, i.e., such channels possess the partial strong converse property. Furthermore, we derive sufficient conditions for the partial strong converse property to hold for memoryless but non-stationary symmetric and degraded wiretap channels. Our proof techniques leverage the information spectrum method, which allows us to establish a necessary and sufficient condition for the partial strong converse to hold for general wiretap channels without any information stability assumptions.

Francesco Renna, Nicola Laurenti, Stefano Tomasin et al.

Information-theoretic secret-key agreement is perhaps the most practically feasible mechanism that provides unconditional security at the physical layer to date. In this paper, we consider the problem of secret-key agreement by sharing randomness at low power over an orthogonal frequency division multiplexing (OFDM) link, in the presence of an eavesdropper. The low power assumption greatly simplifies the design of the randomness sharing scheme, even in a fading channel scenario. We assess the performance of the proposed system in terms of secrecy key rate and show that a practical approach to key sharing is obtained by using low-density parity check (LDPC) codes for information reconciliation. Numerical results confirm the merits of the proposed approach as a feasible and practical solution. Moreover, the outage formulation allows to implement secret-key agreement even when only statistical knowledge of the eavesdropper channel is available.