Rodolphe Sepulchre

B. Mishra, K. Adithya Apuroop, R. Sepulchre · microsoft-research

We propose a new Riemannian geometry for fixed-rank matrices that is specifically tailored to the low-rank matrix completion problem. Exploiting the degree of freedom of a quotient space, we tune the metric on our search space to the particular least square cost function. At one level, it illustrates in a novel way how to exploit the versatile framework of optimization on quotient manifold. At another level, our algorithm can be considered as an improved version of LMaFit, the state-of-the-art Gauss-Seidel algorithm. We develop necessary tools needed to perform both first-order and second-order optimization. In particular, we propose gradient descent schemes (steepest descent and conjugate gradient) and trust-region algorithms. We also show that, thanks to the simplicity of the cost function, it is numerically cheap to perform an exact linesearch given a search direction, which makes our algorithms competitive with the state-of-the-art on standard low-rank matrix completion instances.

Fulvio Forni, Rodolphe Sepulchre

Lyapunov's second theorem is an essential tool for stability analysis of differential equations. The paper provides an analog theorem for incremental stability analysis by lifting the Lyapunov function to the tangent bundle. The Lyapunov function endows the state-space with a Finsler structure. Incremental stability is inferred from infinitesimal contraction of the Finsler metrics through integration along solutions curves.

Silvere Bonnabel, Rodolphe Sepulchre

This paper introduces a new metric and mean on the set of positive semidefinite matrices of fixed-rank. The proposed metric is derived from a well-chosen Riemannian quotient geometry that generalizes the reductive geometry of the positive cone and the associated natural metric. The resulting Riemannian space has strong geometrical properties: it is geodesically complete, and the metric is invariant with respect to all transformations that preserve angles (orthogonal transformations, scalings, and pseudoinversion). A meaningful approximation of the associated Riemannian distance is proposed, that can be efficiently numerically computed via a simple algorithm based on SVD. The induced mean preserves the rank, possesses the most desirable characteristics of a geometric mean, and is easy to compute.

Fulvio Forni, Rodolphe Sepulchre

High-dimensional systems that have a low-dimensional dominant behavior allow for model reduction and simplified analysis. We use differential analysis to formalize this important concept in a nonlinear setting. We show that dominance can be studied through linear dissipation inequalities and an interconnection theory that closely mimics the classical analysis of stability by means of dissipativity theory. In this approach, stability is seen as the limiting situation where the dominant behavior is 0-dimensional. The generalization opens novel tractable avenues to study multistability through 1-dominance and limit cycle oscillations through 2-dominance.

F. A. Miranda-Villatoro, F. Forni, R. Sepulchre

Frequency domain analysis of linear time-invariant (LTI) systems in feedback with static nonlinearities is a classical and fruitful topic of nonlinear systems theory. We generalize this approach beyond equilibrium stability analysis with the aim of characterizing feedback systems whose asymptotic behavior is low dimensional. We illustrate the theory with a generalization of the circle criterion for the analysis of multistable and oscillatory Lur'e feedback systems.

Rodolphe Sepulchre, Guillaume Drion, Alessio Franci

This chapter revisits the concept of excitability, a basic system property of neurons. The focus is on excitable systems regarded as behaviors rather than dynamical systems. By this we mean open systems modulated by specific interconnection properties rather than closed systems classified by their parameter ranges. Modeling, analysis, and synthesis questions can be formulated in the classical language of circuit theory. The input-output characterization of excitability is in terms of the local sensitivity of the current-voltage relationship. It suggests the formulation of novel questions for non-linear system theory, inspired by questions from experimental neurophysiology.

Silvere Bonnabel, Rodolphe Sepulchre

An important property of the Kalman filter is that the underlying Riccati flow is a contraction for the natural metric of the cone of symmetric positive definite matrices. The present paper studies the geometry of a low-rank version of the Kalman filter. The underlying Riccati flow evolves on the manifold of fixed rank symmetric positive semidefinite matrices. Contraction properties of the low-rank flow are studied by means of a suitable metric recently introduced by the authors.

Lek-Heng Lim, Rodolphe Sepulchre, Ke Ye

We show how the Riemannian distance on $\mathbb{S}^n_{++}$, the cone of $n\times n$ real symmetric or complex Hermitian positive definite matrices, may be used to naturally define a distance between two such matrices of different dimensions. Given that $\mathbb{S}^n_{++}$ also parameterizes $n$-dimensional ellipsoids, and inner products on $\mathbb{R}^n$, $n \times n$ covariance matrices of nondegenerate probability distributions, this gives us a natural way to define a geometric distance between a pair of such objects of different dimensions.

Fulvio Forni, Rodolphe Sepulchre

We revisit the classical dissipativity theorem of linear-quadratic theory in a generalized framework where the quadratic storage is negative definite in a p-dimensional subspace and positive definite in a complementary subspace. The classical theory assumes p = 0 and provides an inter- connection theory for stability analysis, i.e. convergence to a zero dimensional attractor. The generalized theory is shown to provide an interconnection theory for p-dominance analysis, i.e. convergence to a p-dimensional dominant subspace. In turn, this property is the differential characterization of a generalized contraction property for nonlinear systems. The proposed generalization opens a novel avenue for the analysis of interconnected systems with low-dimensional attractors.

Fulvio Forni, Raphael M. Jungers, Rodolphe Sepulchre

The notion of path-complete positivity is introduced as a way to generalize the property of positivity from one LTI system to a family of switched LTI systems whose switching rule is constrained by a finite automaton. The generalization builds upon the analogy between stability and positivity, the former referring to the contraction of a norm, the latter referring to the contraction of a cone (or, equivalently, a projective norm). We motivate and investigate the potential of path-positivity and we propose an algorithm for the automatic verification of positivity.

Giacomo Baggio, Rodolphe Sepulchre

This paper revisits the definition of linear time-invariant (LTI) stochastic process within a behavioral systems framework. Building on [Willems, 2013], we derive a canonical representation of an LTI stochastic process and a physically grounded notion of interconnection between independent stochastic processes. We use this framework to analyze the invariance properties enjoyed by distances between spectral densities of LTI processes.

Loris Mendolia, Chenxi Wen, Elisabetta Chicca et al.

Neuromorphic engineering makes use of mixed-signal analog and digital circuits to directly emulate the computational principles of biological brains. Such electronic systems offer a high degree of adaptability, robustness, and energy efficiency across a wide range of tasks, from edge computing to robotics. Within this context, we investigate a key feature of biological neurons: their ability to carry out robust and reliable computation by adapting their input responses and spiking patterns to context through neuromodulation. Achieving analogous levels of robustness and adaptation in neuromorphic circuits through modulatory mechanisms is a largely unexplored path. We present a novel current-mode neuron design that supports robust neuromodulation with minimal model complexity, compatible with standard CMOS technologies. We first introduce a mathematical model of the circuit and provide tools to analyze and tune the neuron behavior; we then demonstrate both theoretically and experimentally the biologically plausible neuromodulation adaptation capabilities of the circuit over a wide range of parameters. All theoretical predictions were verified in experiments on a low-power 180 nm CMOS implementation of the proposed neuron circuit. Due to the analog underlying feedback structure, the proposed adaptive neuromodulable neuron exhibits a high degree of robustness, flexibility, and scalability across operating ranges of currents and temperatures, making it a perfect candidate for real-world neuromorphic applications.

Alberto Padoan, Fulvio Forni, Rodolphe Sepulchre

The paper introduces notions of robustness margins geared towards the analysis and design of systems that switch and oscillate. While such phenomena are ubiquitous in nature and in engineering, a theory of robustness for behaviors away from equilibria is lacking. The proposed framework addresses this need in the framework of p-dominance theory, which aims at generalizing stability theory for the analysis of systems with low-dimensional attractors. Dominance margins are introduced as natural generalisations of stability margins in the context of p-dominance analysis. In analogy with stability margins, dominance margins are shown to admit simple interpretations in terms of familiar frequency domain tools and to provide quantitative measures of robustness for multistable and oscillatory behaviors in Lure systems. The theory is illustrated by means of an elementary mechanical example.

Felix A. Miranda-Villatoro, Fulvio Forni, Rodolphe Sepulchre

The paper extends the concepts of dominance and p-dissipativity to the non-smooth family of linear complementarity systems. Dominance generalizes incremental stability whereas p-dissipativity generalizes incremental passivity. The generalization aims at an interconnection theory for the design and analysis of switching and oscillatory systems. The approach is illustrated by a detailed study of classical electrical circuits that switch and oscillate.

Felix A. Miranda-Villatoro, Fulvio Forni, Rodolphe Sepulchre

The concept of passivity is central to analyze circuits as interconnections of passive components. We illustrate that when used differentially, the same concept leads to an interconnection theory for electrical circuits that switch and oscillate as interconnections of passive components with operational amplifiers (op-amps). The approach builds on recent results on dominance and p-passivity aimed at generalizing dissipativity theory to the analysis of non-equilibrium nonlinear systems. Our paper shows how those results apply to basic and well-known nonlinear circuit architectures. They illustrate the potential of dissipativity theory to design and analyze switching and oscillating circuits quantitatively, very much like their linear counterparts.

Fulvio Forni, Alexandre Mauroy, Rodolphe Sepulchre

The paper shows that normally hyperbolic one-dimensional compact attractors of smooth dynamical systems are characterized by differential positivity, that is, the pointwise infinitesimal contraction of a smooth cone field. The result is analog to the characterization of zero-dimensional hyperbolic attractors by differential stability, which is the pointwise infinitesimal contraction of a Riemannian metric.

Cyrus Mostajeran, Nathaël Da Costa, Graham Van Goffrier et al.

We present a geometric framework for the processing of SPD-valued data that preserves subspace structures and is based on the efficient computation of extreme generalized eigenvalues. This is achieved through the use of the Thompson geometry of the semidefinite cone. We explore a particular geodesic space structure in detail and establish several properties associated with it. Finally, we review a novel inductive mean of SPD matrices based on this geometry.

Yongkang Huo, Fuvio Forni, Rodolphe Sepulchre

We present a novel framework for central pattern generator design that leverages the intrinsic rebound excitability of neurons in combination with winner-takes-all computation. Our approach unifies decision-making and rhythmic pattern generation within a simple yet powerful network architecture that employs all-to-all inhibitory connections enhanced by designable excitatory interactions. This design offers significant advantages regarding ease of implementation, adaptability, and robustness. We demonstrate its efficacy through a ring oscillator model, which exhibits adaptive phase and frequency modulation, making the framework particularly promising for applications in neuromorphic systems and robotics.

Tai Miyazaki Kirby, Luka Ribar, Rodolphe Sepulchre

Analog, low-voltage electronics show great promise in producing silicon neurons (SiNs) with unprecedented levels of energy efficiency. Yet, their inherently high susceptibility to process, voltage and temperature (PVT) variations, and noise has long been recognised as a major bottleneck in developing effective neuromorphic solutions. Inspired by spike transmission studies in biophysical, neocortical neurons, we demonstrate that the inherent noise and variability can coexist with reliable spike transmission in analog SiNs, similarly to biological neurons. We illustrate this property on a recent neuromorphic model of a bursting neuron by showcasing three different relevant types of reliable event transmission: single spike transmission, burst transmission, and the on-off control of a half-centre oscillator (HCO) network.

Thiago B. Burghi, Maarten Schoukens, Rodolphe Sepulchre

After sixty years of quantitative biophysical modeling of neurons, the identification of neuronal dynamics from input-output data remains a challenging problem, primarily due to the inherently nonlinear nature of excitable behaviors. By reformulating the problem in terms of the identification of an operator with fading memory, we explore a simple approach based on a parametrization given by a series interconnection of Generalized Orthonormal Basis Functions (GOBFs) and static Artificial Neural Networks. We show that GOBFs are particularly well-suited to tackle the identification problem, and provide a heuristic for selecting GOBF poles which addresses the ultra-sensitivity of neuronal behaviors. The method is illustrated on the identification of a bursting model from the crab stomatogastric ganglion.

Luka Ribar, Rodolphe Sepulchre

Neuromorphic engineering is a rapidly developing field that aims to take inspiration from the biological organization of neural systems to develop novel technology for computing, sensing, and actuating. The unique properties of such systems call for new signal processing and control paradigms. The article introduces the mixed feedback organization of excitable neuronal systems, consisting of interlocked positive and negative feedback loops acting in distinct timescales. The principles of biological neuromodulation suggest a methodology for designing and controlling mixed-feedback systems neuromorphically. The proposed design consists of a parallel interconnection of elementary circuit elements that mirrors the organization of biological neurons and utilizes the hardware components of neuromorphic electronic circuits. The interconnection structure endows the neuromorphic systems with a simple control methodology that reframes the neuronal control as an input-output shaping problem. The potential of neuronal control is illustrated on elementary network examples that suggest the scalability of the mixed-feedback principles.

Bamdev Mishra, Rodolphe Sepulchre

The paper looks at a scaled variant of the stochastic gradient descent algorithm for the matrix completion problem. Specifically, we propose a novel matrix-scaling of the partial derivatives that acts as an efficient preconditioning for the standard stochastic gradient descent algorithm. This proposed matrix-scaling provides a trade-off between local and global second order information. It also resolves the issue of scale invariance that exists in matrix factorization models. The overall computational complexity is linear with the number of known entries, thereby extending to a large-scale setup. Numerical comparisons show that the proposed algorithm competes favorably with state-of-the-art algorithms on various different benchmarks.

Raphaël Liégeois, Bamdev Mishra, Mattia Zorzi et al.

This paper considers the problem of identifying multivariate autoregressive (AR) sparse plus low-rank graphical models. Based on the corresponding problem formulation recently presented, we use the alternating direction method of multipliers (ADMM) to efficiently solve it and scale it to sizes encountered in neuroimaging applications. We apply this decomposition on synthetic and real neuroimaging datasets with a specific focus on the information encoded in the low-rank structure of our model. In particular, we illustrate that this information captures the spatio-temporal structure of the original data, generalizing classical component analysis approaches.

Fulvio Forni, Rodolphe Sepulchre

The paper introduces and studies differentially positive systems, that is, systems whose linearization along an arbitrary trajectory is positive. A generalization of Perron Frobenius theory is developed in this differential framework to show that the property induces a (conal) order that strongly constrains the asymptotic behavior of solutions. The results illustrate that behaviors constrained by local order properties extend beyond the well-studied class of linear positive systems and monotone systems, which both require a constant cone field and a linear state space.

Nicolas Boumal, Bamdev Mishra, P. -A. Absil et al.

Optimization on manifolds is a rapidly developing branch of nonlinear optimization. Its focus is on problems where the smooth geometry of the search space can be leveraged to design efficient numerical algorithms. In particular, optimization on manifolds is well-suited to deal with rank and orthogonality constraints. Such structured constraints appear pervasively in machine learning applications, including low-rank matrix completion, sensor network localization, camera network registration, independent component analysis, metric learning, dimensionality reduction and so on. The Manopt toolbox, available at www.manopt.org, is a user-friendly, documented piece of software dedicated to simplify experimenting with state of the art Riemannian optimization algorithms. We aim particularly at reaching practitioners outside our field.

B. Mishra, R. Sepulchre

We exploit the versatile framework of Riemannian optimization on quotient manifolds to develop R3MC, a nonlinear conjugate-gradient method for low-rank matrix completion. The underlying search space of fixed-rank matrices is endowed with a novel Riemannian metric that is tailored to the least-squares cost. Numerical comparisons suggest that R3MC robustly outperforms state-of-the-art algorithms across different problem instances, especially those that combine scarcely sampled and ill-conditioned data.

B. Mishra, G. Meyer, R. Sepulchre

This paper addresses the problem of low-rank distance matrix completion. This problem amounts to recover the missing entries of a distance matrix when the dimension of the data embedding space is possibly unknown but small compared to the number of considered data points. The focus is on high-dimensional problems. We recast the considered problem into an optimization problem over the set of low-rank positive semidefinite matrices and propose two efficient algorithms for low-rank distance matrix completion. In addition, we propose a strategy to determine the dimension of the embedding space. The resulting algorithms scale to high-dimensional problems and monotonically converge to a global solution of the problem. Finally, numerical experiments illustrate the good performance of the proposed algorithms on benchmarks.

Anne Collard, Silvère Bonnabel, Christophe Phillips et al.

Statistical analysis of Diffusion Tensor Imaging (DTI) data requires a computational framework that is both numerically tractable (to account for the high dimensional nature of the data) and geometric (to account for the nonlinear nature of diffusion tensors). Building upon earlier studies that have shown that a Riemannian framework is appropriate to address these challenges, the present paper proposes a novel metric and an accompanying computational framework for DTI data processing. The proposed metric retains the geometry and the computational tractability of earlier methods grounded in the affine invariant metric. In addition, and in contrast to earlier methods, it provides an interpolation method which preserves anisotropy, a central information carried by diffusion tensor data.

B. Mishra, G. Meyer, S. Bonnabel et al.

Motivated by the problem of learning a linear regression model whose parameter is a large fixed-rank non-symmetric matrix, we consider the optimization of a smooth cost function defined on the set of fixed-rank matrices. We adopt the geometric framework of optimization on Riemannian quotient manifolds. We study the underlying geometries of several well-known fixed-rank matrix factorizations and then exploit the Riemannian quotient geometry of the search space in the design of a class of gradient descent and trust-region algorithms. The proposed algorithms generalize our previous results on fixed-rank symmetric positive semidefinite matrices, apply to a broad range of applications, scale to high-dimensional problems and confer a geometric basis to recent contributions on the learning of fixed-rank non-symmetric matrices. We make connections with existing algorithms in the context of low-rank matrix completion and discuss relative usefulness of the proposed framework. Numerical experiments suggest that the proposed algorithms compete with the state-of-the-art and that manifold optimization offers an effective and versatile framework for the design of machine learning algorithms that learn a fixed-rank matrix.