Burak Çakmak

Hao Yan, Lei Liu, Yuhao Liu et al.

This paper proposes a coding framework for capacity-region-achieving sparse regression (SR) codes over MIMO multiple-access channels (MIMO-MAC), where a single SR code is used for each user at the transmitter. With random semi-unitary dictionary matrices applied for encoding, multiple-access OAMP (MA-OAMP) enables reliable parallel interference cancellation (PIC) at the receiver. Theoretically, an optimal coding principle with the MA-OAMP receiver, which achieves the sum capacity and, in combination with time sharing, achieves the entire capacity region, is established as the guiding principle for designing capacity-region-achieving codes. Accordingly, a coding scheme for capacity-region-achieving SR codes is proposed via proper power allocation over the position-modulated signals.

Burak Çakmak, Hao Yan, Alexander Fengler et al.

We solve the open problem of constructing a Bayes-optimal iterative signal recovery algorithm for linear-Gaussian \emph{multiuser} communication systems with random precoding at the transmitters.Specifically, we consider the received signal model $\mathbf{y} = \sum_{u} \mathbf{H}_u \mathbfΞ_u \mathbf{s}_u + \mathbf{n}$, where $\mathbf{n}$ is white Gaussian noise, $\{\mathbf{H}_u \in \mathbb{C}^{L \times L}\}$ are discrete-time channel matrices -- modeling a wide class of generally time-varying and dispersive linear channels with possibly multiple antennas -- and the precoding matrices $\{\boldsymbolΞ_u \in \mathbb{C}^{L \times N_u}\}$ are drawn independently from a right-unitarily invariant random matrix ensemble. We consider generic \emph{non-separable} (coded) systems where the users' signals $\{\mathbf{s}_u\}$ follow general (non-factorizing) distributions. For this model, we introduce a novel orthogonal/vector approximate message passing (OAMP/VAMP)-type framework, including an algorithm and its high-dimensional (but finite-sample) analysis. From an algorithmic standpoint, the proposed method can be interpreted as an \emph{interpolation} between Minka's expectation propagation (EP)--a widely used method in machine learning--and OAMP. Our main theoretical contribution is the explicit finite-sample analysis of the proposed algorithm. Furthermore, we analyze the associated inference problem via a replica-symmetric (RS) ansatz by using a novel disorder-averaging technique. Both the (rigorous) high-dimensional analysis of the algorithm and the RS ansatz reveal the same decoupling principle, establishing that the proposed algorithm is asymptotically Bayes-optimal under the validity of the RS ansatz.

Burak Çakmak, Yue M. Lu, Manfred Opper

Motivated by the recent application of approximate message passing (AMP) to the analysis of convex optimizations in multi-class classifications [Loureiro, et. al., 2021], we present a convergence analysis of AMP dynamics with non-separable multivariate nonlinearities. As an application, we present a complete (and independent) analysis of the motivated convex optimization problem.

Burak Çakmak, Yue M. Lu, Manfred Opper

We analyze the dynamics of a random sequential message passing algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. Moreover, we consider a model mismatching setting, where the teacher model and the one used by the student may be different. By means of dynamical functional approach, we obtain exact dynamical mean-field equations characterizing the dynamics of the inference algorithm. We also derive a range of model parameters for which the sequential algorithm does not converge. The boundary of this parameter range coincides with the de Almeida Thouless (AT) stability condition of the replica symmetric ansatz for the static probabilistic model.

Burak Çakmak, Manfred Opper

We analyze the random sequential dynamics of a message passing algorithm for Ising models with random interactions in the large system limit. We derive exact results for the two-time correlation functions and the speed of convergence. The {\em de Almedia-Thouless} stability criterion of the static problem is found to be necessary and sufficient for the global convergence of the random sequential dynamics.

Manfred Opper, Burak Çakmak

We use freeness assumptions of random matrix theory to analyze the dynamical behavior of inference algorithms for probabilistic models with dense coupling matrices in the limit of large systems. For a toy Ising model, we are able to recover previous results such as the property of vanishing effective memories and the analytical convergence rate of the algorithm.

Burak Çakmak, Manfred Opper

We propose an iterative algorithm for solving the Thouless-Anderson-Palmer (TAP) equations of Ising models with arbitrary rotation invariant (random) coupling matrices. In the thermodynamic limit, we prove by means of the dynamical functional method that the proposed algorithm converges when the so-called de Almeida Thouless (AT) criterion is fulfilled. Moreover, we give exact analytical expressions for the rate of the convergence.

Burak Çakmak, Manfred Opper

We study asymptotic properties of expectation propagation (EP) -- a method for approximate inference originally developed in the field of machine learning. Applied to generalized linear models, EP iteratively computes a multivariate Gaussian approximation to the exact posterior distribution. The computational complexity of the repeated update of covariance matrices severely limits the application of EP to large problem sizes. In this study, we present a rigorous analysis by means of free probability theory that allows us to overcome this computational bottleneck if specific data matrices in the problem fulfill certain properties of asymptotic freeness. We demonstrate the relevance of our approach on the gene selection problem of a microarray dataset.

Burak Çakmak, Manfred Opper, Bernard H. Fleury et al.

We investigate the problem of approximate Bayesian inference for a general class of observation models by means of the expectation propagation (EP) framework for large systems under some statistical assumptions. Our approach tries to overcome the numerical bottleneck of EP caused by the inversion of large matrices. Assuming that the measurement matrices are realizations of specific types of ensembles we use the concept of freeness from random matrix theory to show that the EP cavity variances exhibit an asymptotic self-averaging property. They can be pre-computed using specific generating functions, i.e. the R- and/or S-transforms in free probability, which do not require matrix inversions. Our approach extends the framework of (generalized) approximate message passing -- assumes zero-mean iid entries of the measurement matrix -- to a general class of random matrix ensembles. The generalization is via a simple formulation of the R- and/or S-transforms of the limiting eigenvalue distribution of the Gramian of the measurement matrix. We demonstrate the performance of our approach on a signal recovery problem of nonlinear compressed sensing and compare it with that of EP.