Augusto Ferrante

Augusto Ferrante, Chiara Masiero, Michele Pavon

The concept of spectral relative entropy rate is introduced for jointly stationary Gaussian processes. Using classical information-theoretic results, we establish a remarkable connection between time and spectral domain relative entropy rates. This naturally leads to a new spectral estimation technique where a multivariate version of the Itakura-Saito distance is employed}. It may be viewed as an extension of the approach, called THREE, introduced by Byrnes, Georgiou and Lindquist in 2000 which, in turn, followed in the footsteps of the Burg-Jaynes Maximum Entropy Method. Spectral estimation is here recast in the form of a constrained spectrum approximation problem where the distance is equal to the processes relative entropy rate. The corresponding solution entails a complexity upper bound which improves on the one so far available in the multichannel framework. Indeed, it is equal to the one featured by THREE in the scalar case. The solution is computed via a globally convergent matricial Newton-type algorithm. Simulations suggest the effectiveness of the new technique in tackling multivariate spectral estimation tasks, especially in the case of short data records.

Francesca P. Carli, Augusto Ferrante, Michele Pavon et al.

This paper deals with maximum entropy completion of partially specified block-circulant matrices. Since positive definite symmetric circulants happen to be covariance matrices of stationary periodic processes, in particular of stationary reciprocal processes, this problem has applications in signal processing, in particular to image modeling. In fact it is strictly related to maximum likelihood estimation of bilateral AR-type representations of acausal signals subject to certain conditional independence constraints. The maximum entropy completion problem for block-circulant matrices has recently been solved by the authors, although leaving open the problem of an efficient computation of the solution. In this paper, we provide an effcient algorithm for computing its solution which compares very favourably with existing algorithms designed for positive definite matrix extension problems. The proposed algorithm benefits from the analysis of the relationship between our problem and the band-extension problem for block-Toeplitz matrices also developed in this paper.

Francesca Carli, Augusto Ferrante, Michele Pavon et al.

Stationary reciprocal processes defined on a finite interval of the integer line can be seen as a special class of Markov random fields restricted to one dimension. Non stationary reciprocal processes have been extensively studied in the past especially by Jamison, Krener, Levy and co-workers. The specialization of the non-stationary theory to the stationary case, however, does not seem to have been pursued in sufficient depth in the literature. Stationary reciprocal processes (and reciprocal stochastic models) are potentially useful for describing signals which naturally live in a finite region of the time (or space) line. Estimation or identification of these models starting from observed data seems still to be an open problem which can lead to many interesting applications in signal and image processing. In this paper, we discuss a class of reciprocal processes which is the acausal analog of auto-regressive (AR) processes, familiar in control and signal processing. We show that maximum likelihood identification of these processes leads to a covariance extension problem for block-circulant covariance matrices. This generalizes the famous covariance band extension problem for stationary processes on the integer line. As in the usual stationary setting on the integer line, the covariance extension problem turns out to be a basic conceptual and practical step in solving the identification problem. We show that the maximum entropy principle leads to a complete solution of the problem.

Augusto Ferrante, Giorgio Picci

Matrix spectral factorization is traditionally described as finding spectral factors having a fixed analytic pole configuration. The classification of spectral factors then involves studying the solutions of a certain algebraic Riccati equation which parametrizes their zero structure. The pole structure of the spectral factors can be also parametrized in terms of solutions of another Riccati equation. We study the relation between the solution sets of these two Riccati equations and describe the construction of general spectral factors which involve both zero- and pole-flipping on an arbitrary reference spectral factor.

Riccardo Zattra, Giacomo Baggio, Umberto Casti et al.

Transformers, powered by the attention mechanism, are the backbone of most foundation models, yet they suffer from quadratic complexity and difficulties in dealing with long-range dependencies in the input sequence. Recent work has shown that state space models (SSMs) provide an efficient alternative, with the S6 module at the core of the Mamba architecture achieving state-of-the-art results on long-sequence benchmarks. In this paper, we introduce the COFFEE (COntext From FEEdback) model, a novel time-varying SSM that incorporates state feedback to enable context-dependent selectivity, while still allowing for parallel implementation. Whereas the selectivity mechanism of S6 only depends on the current input, COFFEE computes it from the internal state, which serves as a compact representation of the sequence history. This shift allows the model to regulate its dynamics based on accumulated context, improving its ability to capture long-range dependencies. In addition to state feedback, we employ an efficient model parametrization that removes redundancies present in S6 and leads to a more compact and trainable formulation. On the induction head task, COFFEE achieves near-perfect accuracy with two orders of magnitude fewer parameters and training sequences compared to S6. On MNIST, COFFEE largely outperforms S6 within the same architecture, reaching 97% accuracy with only 3585 parameters. These results showcase the role of state feedback as a key mechanism for building scalable and efficient sequence models.

Augusto Ferrante, Nicola Laurenti, Chiara Masiero et al.

For a physical layer message authentication procedure based on the comparison of channel estimates obtained from the received messages, we focus on an outer bound on the type I/II error probability region. Channel estimates are modelled as multivariate Gaussian vectors, and we assume that the attacker has only some side information on the channel estimate, which he does not know directly. We derive the attacking strategy that provides the tightest bound on the error region, given the statistics of the side information. This turns out to be a zero mean, circularly symmetric Gaussian density whose correlation matrices may be obtained by solving a constrained optimization problem. We propose an iterative algorithm for its solution: Starting from the closed form solution of a relaxed problem, we obtain, by projection, an initial feasible solution; then, by an iterative procedure, we look for the fixed point solution of the problem. Numerical results show that for cases of interest the iterative approach converges, and perturbation analysis shows that the found solution is a local minimum.