Abbas El Gamal

Han-I Su, Abbas El Gamal

The high variability of renewable energy resources presents significant challenges to the operation of the electric power grid. Conventional generators can be used to mitigate this variability but are costly to operate and produce carbon emissions. Energy storage provides a more environmentally friendly alternative, but is costly to deploy in large amounts. This paper studies the limits on the benefits of energy storage to renewable energy: How effective is storage at mitigating the adverse effects of renewable energy variability? How much storage is needed? What are the optimal control policies for operating storage? To provide answers to these questions, we first formulate the power flow in a single-bus power system with storage as an infinite horizon stochastic program. We find the optimal policies for arbitrary net renewable generation process when the cost function is the average conventional generation (environmental cost) and when it is the average loss of load probability (reliability cost). We obtain more refined results by considering the multi-timescale operation of the power system. We view the power flow in each timescale as the superposition of a predicted (deterministic) component and an prediction error (residual) component and formulate the residual power flow problem as an infinite horizon dynamic program. Assuming that the net generation prediction error is an IID process, we quantify the asymptotic benefits of storage. With the additional assumption of Laplace distributed prediction error, we obtain closed form expressions for the stationary distribution of storage and conventional generation. Finally, we propose a two-threshold policy that trades off conventional generation saving with loss of load probability. We illustrate our results and corroborate the IID and Laplace assumptions numerically using datasets from CAISO and NREL.

Thomas Navidi, Abbas El Gamal, Ram Rajagopal

This paper presents a two-layer distributed energy resource (DER) coordination architecture that allows for separate ownership of data, operates with data subjected to a large buffering delay, and employs a new measure of power quality. The two-layer architecture comprises a centralized model predictive controller (MPC) and several decentralized MPCs each operating independently with no direct communication between them and with infrequent communication with the centralized controller. The goal is to minimize a combination of total energy cost and a measure of power quality while obeying cyber-physical constraints. The global controller utilizes a fast AC optimal power flow (OPF) solver and extensive parallelization to scale the solution to large networks. Each local controller attempts to maximize arbitrage profit while following the load profile and constraints dictated by the global controller. Extensive simulations are performed for two distribution networks under a wide variety of possible storage and solar penetrations enabled by the controller speed. The simulations show that (i) the two-layer architecture can achieve tenfold improvement in power quality relative to no coordination, while capturing nearly all of the available arbitrage profit for a moderate amount of storage penetration, and (ii) both power quality and arbitrage profits are optimized when the solar and storage are distributed more widely over the network, hence it is more effective to install storage closer to the consumer.

Abbas El Gamal, Maxim Raginsky

Information theory plays a central role in establishing fundamental limits on what any learning or estimation algorithm can -- and cannot -- achieve, regardless of computational power. In this chapter, we provide an introduction to these connections. End-of-chapter exercises makes the material suitable for both classroom use and self-study. We begin by introducing concentration inequalities along with the notions of covering and packing in metric spaces, and the associated concept of metric entropy. These tools are essential for our analysis. We then introduce the learning-theoretic framework and derive upper bounds on generalization error in terms of metric entropy, Rademacher complexity, and the VC dimension, as well as mutual information and relative entropy. Finally we discuss the minimax estimation framework and establish lower bounds on minimax risk using Fano's inequality, yielding bounds in terms of relative entropy and covering and packing numbers. This manuscript contains preprint of a chapter under consideration for inclusion in the forthcoming third edition of Cover and Thomas's Elements of Information Theory, posted with permission from Wiley. It would follow the chapter posted at arXiv:2605.02989 . The table of contents of the new edition can be found at: https://docs.google.com/document/d/1L-m4oQEJw1PJhoxBeMwrrBD8S_HmvzMEkPbYvS24980/edit?usp=sharing . For feedback, please contact abbas@ee.stanford.edu.

Abbas El Gamal

This manuscript contains preprint of a chapter under consideration for inclusion in the forthcoming third edition of {\em Cover and Thomas's Elements of Information Theory}, posted with permission from Wiley. The table of contents EIT-3 ToC of the new edition can be found at: https://docs.google.com/document/d/1L-m4oQEJw1PJhoxBeMwrrBD8S_HmvzMEkPbYvS24980/edit?usp=sharing . For feedback, please contact abbas@ee.stanford.edu Learning and information theory intersect in both model training and the characterization of fundamental performance limits. This manuscript provides a concise and accessible treatment of the first intersection, requiring only basic background in information theory and statistics at the senior undergraduate or first-year graduate level. End-of-chapter exercises make the material well suited for classroom use as well as self-study. The chapter focuses on the role of divergence measures in model training, with examples ranging from linear and logistic regression to autoregressive models, variational autoencoders, diffusion models, generative adversarial networks, and score-based models. It introduces the evidence lower bound (ELBO), $f$\!-divergences, and the Fisher divergence. In particular, the treatment of the generative diffusion model provides a more systematic and explicit derivation than is typical in the literature.

Hongchao Zhou, Abbas El Gamal

Shannon showed that to achieve perfect secrecy in point-to-point communication, the message rate cannot exceed the shared secret key rate giving rise to the simple one-time pad encryption scheme. In this paper, we extend this work from point-to-point to networks. We consider a connected network with pairwise communication between the nodes. We assume that each node is provided with a certain amount of secret bits before communication commences. An eavesdropper with unlimited computing power has access to all communication and can hack a subset of the nodes not known to the rest of the nodes. We investigate the limits on information-theoretic secure communication for this network. We establish a tradeoff between the secure channel rate (for a node pair) and the secure network rate (sum over all node pair rates) and show that perfect secrecy can be achieved if and only if the sum rate of any subset of unhacked channels does not exceed the shared unhacked-secret-bit rate of these channels. We also propose two practical and efficient schemes that achieve a good balance of network and channel rates with perfect secrecy guarantee. This work has a wide range of potential applications for which perfect secrecy is desired, such as cyber-physical systems, distributed-control systems, and ad-hoc networks.

Cheuk Ting Li, Abbas El Gamal

This paper shows that the Hirschfeld-Gebelein-Rényi maximal correlation between the message and the ciphertext provides good secrecy guarantees for cryptosystems that use short keys. We first establish a bound on the eavesdropper's advantage in guessing functions of the message in terms of maximal correlation and the Rényi entropy of the message. This result implies that maximal correlation is stronger than the notion of entropic security introduced by Russell and Wang. We then show that a small maximal correlation $ρ$ can be achieved via a randomly generated cipher with key length $\approx2\log(1/ρ)$, independent of the message length, and by a stream cipher with key length $2\log(1/ρ)+\log n+2$ for a message of length $n$. We establish a converse showing that these ciphers are close to optimal. This is in contrast to entropic security for which there is a gap between the lower and upper bounds. Finally, we show that a small maximal correlation implies secrecy with respect to several mutual information based criteria but is not necessarily implied by them. Hence, maximal correlation is a stronger and more practically relevant measure of secrecy than mutual information.