Eduardo Sontag

Maja Skataric, Eduardo Sontag

An ubiquitous property of biological sensory systems is adaptation: a step increase in stimulus triggers an initial change in a biochemical or physiological response, followed by a more gradual relaxation toward a basal, pre-stimulus level. Adaptation helps maintain essential variables within acceptable bounds and allows organisms to readjust themselves to an optimum and non-saturating sensitivity range when faced with a prolonged change in their environment. Recently, it was shown theoretically and experimentally that many adapting systems, both at the organism and single-cell level, enjoy a remarkable additional feature: scale invariance, meaning that the initial, transient behavior remains (approximately) the same even when the background signal level is scaled. In this work, we set out to investigate under what conditions a broadly used model of biochemical enzymatic networks will exhibit scale-invariant behavior. An exhaustive computational study led us to discover a new property of surprising simplicity and generality, uniform linearizations with fast output (ULFO), whose validity we show is both necessary and sufficient for scale invariance of enzymatic networks. Based on this study, we go on to develop a mathematical explanation of how ULFO results in scale invariance. Our work provides a surprisingly consistent, simple, and general framework for understanding this phenomenon, and results in concrete experimental predictions.

Oren Shoval, Uri Alon, Eduardo Sontag

We study in this paper certain properties of the responses of dynamical systems to external inputs. The motivation arises from molecular systems biology. and, in particular, the recent discovery of an important transient property, related to Weber's law in psychophysics: "fold-change detection" in adapting systems, the property that scale uncertainty does not affect responses. FCD appears to play an important role in key signaling transduction mechanisms in eukaryotes, including the ERK and Wnt pathways, as well as in E.coli and possibly other prokaryotic chemotaxis pathways. In this paper, we provide further theoretical results regarding this property. Far more generally, we develop a necessary and sufficient characterization of adapting systems whose transient behaviors are invariant under the action of a set (often, a group) of symmetries in their sensory field. A particular instance is FCD, which amounts to invariance under the action of the multiplicative group of positive real numbers. Our main result is framed in terms of a notion which extends equivariant actions of compact Lie groups. Its proof relies upon control theoretic tools, and in particular the uniqueness theorem for minimal realizations.

Eron Ristich, Eduardo Sontag, Necmiye Ozay

Understanding when linear immersions of nonlinear dynamical systems exist is important since such immersions allow us to leverage the rich tools of linear system theory to analyze nonlinear dynamics. Recently, Liu et al. (2023) showed that continuous-time dynamical systems that admit countably many but more than one omega-limit sets cannot be immersed into finite dimensional linear systems with a one-to-one and continuous mapping. In this paper, we extend these results to discrete-time dynamics and show that similar obstructions exist also in discrete time. We further consider a generalization involving alpha-limit sets. Several examples are provided to demonstrate the results.

Andreas Oliveira, Arthur C. B. de Oliveira, Mario Sznaier et al.

The Łojasiewicz inequality characterizes objective-value convergence along gradient flows and, in special cases, yields exponential decay of the cost. However, such results do not directly give rates of convergence in the state. In this paper, we use contraction theory to derive state-space guarantees for gradient systems satisfying generalized Łojasiewicz inequalities. We first show that, when the objective has a unique strongly convex minimizer, the generalized Łojasiewicz inequality implies semi-global exponential stability; on arbitrary compact subsets, this yields exponential stability. We then give two curvature-based sufficient conditions, together with constraints on the Łojasiewicz rate, under which the nonconvex gradient flow is globally incrementally exponentially stable.

Arthur Castello Branco de Oliveira, Dhruv Jatkar, Eduardo Sontag

This paper investigates how the compositional structure of neural networks shapes their optimization landscape and training dynamics. We analyze the gradient flow associated with overparameterized optimization problems, which can be interpreted as training a neural network with linear activations. Remarkably, we show that the global convergence properties can be derived for any cost function that is proper and real analytic. We then specialize the analysis to scalar-valued cost functions, where the geometry of the landscape can be fully characterized. In this setting, we demonstrate that key structural features -- such as the location and stability of saddle points -- are universal across all admissible costs, depending solely on the overparameterized representation rather than on problem-specific details. Moreover, we show that convergence can be arbitrarily accelerated depending on the initialization, as measured by an imbalance metric introduced in this work. Finally, we discuss how these insights may generalize to neural networks with sigmoidal activations, showing through a simple example which geometric and dynamical properties persist beyond the linear case.

Joshua Hanson, Maxim Raginsky, Eduardo Sontag

We consider the following learning problem: Given sample pairs of input and output signals generated by an unknown nonlinear system (which is not assumed to be causal or time-invariant), we wish to find a continuous-time recurrent neural net with hyperbolic tangent activation function that approximately reproduces the underlying i/o behavior with high confidence. Leveraging earlier work concerned with matching output derivatives up to a given finite order, we reformulate the learning problem in familiar system-theoretic language and derive quantitative guarantees on the sup-norm risk of the learned model in terms of the number of neurons, the sample size, the number of derivatives being matched, and the regularity properties of the inputs, the outputs, and the unknown i/o map.