Lorenzo Livi

Sajjad Heydari, Stefano Raniolo, Lorenzo Livi et al.

Predicting structural and energetic properties of a molecular system is one of the fundamental tasks in molecular simulations, and it has use cases in chemistry, biology, and medicine. In the past decade, the advent of machine learning algorithms has impacted on molecular simulations for various tasks, including property prediction of atomistic systems. In this paper, we propose a novel methodology for transferring knowledge obtained from simple molecular systems to a more complex one, possessing a significantly larger number of atoms and degrees of freedom. In particular, we focus on the classification of high and low free-energy states. Our approach relies on utilizing (i) a novel hypergraph representation of molecules, encoding all relevant information for characterizing the potential energy of a conformation, and (ii) novel message passing and pooling layers for processing and making predictions on such hypergraph-structured data. Despite the complexity of the problem, our results show a remarkable AUC of 0.92 for transfer learning from tri-alanine to the deca-alanine system. Moreover, we show that the very same transfer learning approach can be used to group, in an unsupervised way, various secondary structures of deca-alanine in clusters having similar free-energy values. Our study represents a proof of concept that reliable transfer learning models for molecular systems can be designed paving the way to unexplored routes in prediction of structural and energetic properties of biologically relevant systems.

Lorenzo Livi

We develop a statistical theory of temporal learnability in recurrent neural networks, quantifying the maximal temporal horizon $\mathcal{H}_N$ over which gradient-based learning can recover lag-dependent structure at finite sample size $N$. The theory is built on the effective learning rate envelope $f(\ell)$, a functional that captures how gating mechanisms and adaptive optimizers jointly shape the coupling between state-space transport and parameter updates during Backpropagation Through Time. Under heavy-tailed ($α$-stable) gradient noise, where empirical averages concentrate at rate $N^{-1/κ_α}$ with $κ_α= α/(α-1)$, the interplay between envelope decay and statistical concentration yields explicit scaling laws for the growth of $\mathcal{H}_N$: logarithmic, polynomial, and exponential temporal learning regimes emerge according to the decay law of $f(\ell)$. These results identify the envelope decay geometry as the key determinant of temporal learnability: slower attenuation of $f(\ell)$ enlarges the learnability window $\mathcal{H}_N$, while heavy-tailed gradient noise compresses temporal horizons by weakening statistical concentration. Experiments across multiple gated architectures and optimizers corroborate these structural predictions.

Lorenzo Livi

We develop a statistical theory of temporal learnability in recurrent neural networks, showing how gating mechanisms determine the learnability window $\mathcal{H}_N$, defined as the maximal temporal horizon over which gradient information remains recoverable at sample size $N$. While classical analyses emphasize numerical stability of Jacobian products, we show that stability alone does not guarantee recoverability. Instead, learnability is governed by the interaction between the decay geometry of the effective learning rate envelope $f(\ell)=\|μ_{t,\ell}\|_1$, derived from first-order expansions of gate-induced Jacobians in Backpropagation Through Time, and the statistical concentration properties of stochastic gradients. Under heavy-tailed ($α$-stable) gradient noise, empirical averages concentrate at rate $N^{-1/κ_α}$ with $κ_α=α/(α-1)$. We prove that this interaction yields explicit scaling laws for the growth of $\mathcal{H}_N$, distinguishing logarithmic, polynomial, and exponential temporal learning regimes according to the attenuation of $f(\ell)$. The theory reveals that gate-induced time-scale spectra are the dominant determinants of temporal learnability: broader spectra slow envelope decay and systematically expand $\mathcal{H}_N$, whereas heavy-tailed noise uniformly compresses temporal horizons by weakening statistical concentration. Empirical results across multiple gated architectures confirm these structural scaling predictions.

Daniele Zambon, Cesare Alippi, Lorenzo Livi

We present Graph Random Neural Features (GRNF), a novel embedding method from graph-structured data to real vectors based on a family of graph neural networks. The embedding naturally deals with graph isomorphism and preserves the metric structure of the graph domain, in probability. In addition to being an explicit embedding method, it also allows us to efficiently and effectively approximate graph metric distances (as well as complete kernel functions); a criterion to select the embedding dimension trading off the approximation accuracy with the computational cost is also provided. GRNF can be used within traditional processing methods or as a training-free input layer of a graph neural network. The theoretical guarantees that accompany GRNF ensure that the considered graph distance is metric, hence allowing to distinguish any pair of non-isomorphic graphs.

Filippo Maria Bianchi, Lorenzo Livi, Alberto Ferrante et al.

We tackle the problem of classifying Electrocardiography (ECG) signals with the aim of predicting the onset of Paroxysmal Atrial Fibrillation (PAF). Atrial fibrillation is the most common type of arrhythmia, but in many cases PAF episodes are asymptomatic. Therefore, in order to help diagnosing PAF, it is important to design procedures for detecting and, more importantly, predicting PAF episodes. We propose a method for predicting PAF events whose first step consists of a feature extraction procedure that represents each ECG as a multi-variate time series. Successively, we design a classification framework based on kernel similarities for multi-variate time series, capable of handling missing data. We consider different approaches to perform classification in the original space of the multi-variate time series and in an embedding space, defined by the kernel similarity measure. We achieve a classification accuracy comparable with state of the art methods, with the additional advantage of detecting the PAF onset up to 15 minutes in advance.