Sérgio Pequito

Sergio Pequito, Soummya Kar, A. Pedro Aguiar

This paper addresses problems on the structural design of control systems taking explicitly into consideration the possible application to large-scale systems. We provide an efficient and unified framework to solve the following major minimization problems: (i) selection of the minimum number of manipulated/measured variables to achieve structural controllability/observability of the system, and (ii) selection of the minimum number of feedback interconnections between measured and manipulated variables such that the closed-loop system has no structurally fixed modes. Contrary to what would be expected, we show that it is possible to obtain a global solution for each of the aforementioned minimization problems using polynomial complexity algorithms in the number of the state variables of the system. In addition, we provide several new graph-theoretic characterizations of structural systems concepts, which, in turn, enable us to characterize all possible solutions to the above problems.

Sergio Pequito, George J. Pappas

This paper addresses a structural design problem in control systems, and explicitly takes into consideration the possible application to large-scale systems. More precisely, we aim to determine and characterize the minimum number of manipulated state variables ensuring structural controllability of switched linear continuous-time systems. Towards this goal, we provide a new necessary and sufficient condition that leverages both graph-theoretic and algebraic properties required to ensure feasibility of the solutions. With this new condition, we show that a solution can be determined by an efficient procedure, i.e., polynomial in the number of state variables. In addition, we also discuss the switching signal properties that ensure structural controllability and the computational complexity of determining these sequences. In particular, we show that determining the minimum number of modes that a switching signal requires to ensure structural controllability is NP-hard.

Sergio Pequito, Soummya Kar, A. Pedro Aguiar

In this paper we address the actuator/sensor allocation problem for linear time invariant (LTI) systems. Given the structure of an autonomous linear dynamical system, the goal is to design the structure of the input matrix (commonly denoted by $B$) such that the system is structurally controllable with the restriction that each input be dedicated, i.e., it can only control directly a single state variable. We provide a methodology that addresses this design question: specifically, we determine the minimum number of dedicated inputs required to ensure such structural controllability, and characterize, and characterizes all (when not unique) possible configurations of the \emph{minimal} input matrix $B$. Furthermore, we show that the proposed solution methodology incurs \emph{polynomial complexity} in the number of state variables. By duality, the solution methodology may be readily extended to the structural design of the corresponding minimal output matrix (commonly denoted by $C$) that ensures structural observability.

Andreea B. Alexandru, Sergio Pequito, Ali Jadbabaie et al.

In this paper, we study the problem of jointly retrieving the state of a dynamical system, as well as the state of the sensors deployed to estimate it. We assume that the sensors possess a simple computational unit that is capable of performing simple operations, such as retaining the current state and model of the system in its memory. We assume the system to be observable (given all the measurements of the sensors), and we ask whether each sub-collection of sensors can retrieve the state of the underlying physical system, as well as the state of the remaining sensors. To this end, we consider communication between neighboring sensors, whose adjacency is captured by a communication graph. We then propose a linear update strategy that encodes the sensor measurements as states in an augmented state space, with which we provide the solution to the problem of retrieving the system and sensor states. The present paper contains three main contributions. First, we provide necessary and sufficient conditions to ensure observability of the system and sensor states from any sensor. Second, we address the problem of adding communication between sensors when the necessary and sufficient conditions are not satisfied, and devise a strategy to this end. Third, we extend the former case to include different costs of communication between sensors. Finally, the concepts defined and the method proposed are used to assess the state of an example of approximate structural brain dynamics through linearized measurements.

Jingqi Li, Ximing Chen, Sérgio Pequito et al.

In this paper, we consider the structural stabilizability problem of undirected networks. More specifically, we are tasked to infer the stabilizability of an undirected network from its underlying topology, where the undirected networks are modeled as continuous-time linear time-invariant (LTI) systems involving symmetric state matrices. Firstly, we derive a graph-theoretic necessary and sufficient condition for structural stabilizability of undirected networks. Then, we propose a method to infer the maximum dimension of stabilizable subspace solely based on the network structure. Based on these results, on one hand, we study the optimal actuator-disabling attack problem, i.e., removing a limited number of actuators to minimize the maximum dimension of stabilizable subspace. We show this problem is NP-hard. On the other hand, we study the optimal recovery problem with respect to the same kind of attacks, i.e., adding a limited number of new actuators such that the maximum dimension of stabilizable subspace is maximized. We prove the optimal recovery problem is also NP-hard, and we develop a (1-1/e) approximation algorithm to this problem.

Xiaofei Liu, Sergio Pequito, Soummya Kar et al.

This paper addresses problems on the robust structural design of complex networks. More precisely, we address the problem of deploying the minimum number of dedicated sensors, i.e., those measuring a single state variable, that ensure the network to be structurally observable under disruptive scenarios. The disruptive scenarios considered are as follows: (i) the malfunction/loss of one arbitrary sensor, and (ii) the failure of connection (either unidirectional or bidirectional communication) between a pair of agents. First, we show these problems to be NP-hard, which implies that efficient algorithms to determine a solution are unlikely to exist. Secondly, we propose an intuitive two step approach: (1) we achieve an arbitrary minimum sensor placement ensuring structural observability; (2) we develop a sequential process to find minimum number of additional sensors required for robust observability. This step can be solved by recasting it as a weighted set covering problem. Although this is known to be an NP-hard problem, feasible approximations can be determined in polynomial-time that can be used to obtain feasible approximations to the robust structural design problems with optimality guarantees.

Sergio Pequito, Farshad Khorrami, Prashanth Krishnamurthy et al.

This paper considers the analysis and design of resilient/robust decentralized control systems. Specifically, we aim to assess how the pairing of sensors and actuators lead to architectures that are resilient to attacks/hacks for industrial control systems and other complex cyber-physical systems. We consider inherent structural properties such as internal fixed modes of a dynamical system depending on actuation, sensing, and interconnection/communication structure for linear discrete time-invariant dynamical systems. We introduce the notion of resilient fixed-modes free system that ensures the non-existence of fixed modes when the actuation-sensing-communication structure is compromised due to attacks by a malicious agent on actuators, sensors, or communication components and natural failures. Also, we provide a graph-theoretical characterization for the resilient structurally fixed modes that enables to capture the non-existence of resilient fixed modes for almost all possible systems' realizations. Additionally, we address the minimum actuation-sensing-communication co-design ensuring the non-existence of resiliently structurally fixed modes, which we show to be NP-hard. Notwithstanding, we identify conditions that are often satisfied in engineering settings and under which the co-design problem is solvable in polynomial-time complexity. Furthermore, we leverage the structural insights and properties to provide a convex optimization method to design the gain for a parametrized system and satisfying the sparsity of a given information pattern. Thus, exploring the interplay between structural and non-structural systems to ensure their resilience. Finally, the efficacy of the proposed approach is demonstrated on a power grid example.

Sarthak Chatterjee, Orlando Romero, Sérgio Pequito

Closed-loop neurotechnology requires the capability to predict the state evolution and its regulation under (possibly) partial measurements. There is evidence that neurophysiological dynamics can be modeled by fractional-order dynamical systems. Therefore, we propose to establish a separation principle for discrete-time fractional-order dynamical systems, which are inherently nonlinear and are able to capture spatiotemporal relations that exhibit non-Markovian properties. The separation principle states that the problems of controller and state estimator design can be done independently of each other while ensuring proper estimation and control in closed-loop setups. Lastly, we illustrate, as proof-of-concept, the application of the separation principle when designing controllers and estimators for these classes of systems in the context of neurophysiological data. In particular, we rely on real data to derive the models used to assess and regulate the evolution of closed-loop neurotechnologies based on electroencephalographic data.

Mohammadreza Doostmohammadian, Sérgio Pequito

Distributed optimization finds many applications in machine learning, signal processing, and control systems. In these real-world applications, the constraints of communication networks, particularly limited bandwidth, necessitate implementing quantization techniques. In this paper, we propose distributed optimization dynamics over multi-agent networks subject to logarithmically quantized data transmission. Under this condition, data exchange benefits from representing smaller values with more bits and larger values with fewer bits. As compared to uniform quantization, this allows for higher precision in representing near-optimal values and more accuracy of the distributed optimization algorithm. The proposed optimization dynamics comprise a primary state variable converging to the optimizer and an auxiliary variable tracking the objective function's gradient. Our setting accommodates dynamic network topologies, resulting in a hybrid system requiring convergence analysis using matrix perturbation theory and eigenspectrum analysis.

Andrei Lixandru, Marcel van Gerven, Sergio Pequito

Distributed optimization is fundamental to modern machine learning applications like federated learning, but existing methods often struggle with ill-conditioned problems and face stability-versus-speed tradeoffs. We introduce fractional order distributed optimization (FrODO); a theoretically-grounded framework that incorporates fractional-order memory terms to enhance convergence properties in challenging optimization landscapes. Our approach achieves provable linear convergence for any strongly connected network. Through empirical validation, our results suggest that FrODO achieves up to 4 times faster convergence versus baselines on ill-conditioned problems and 2-3 times speedup in federated neural network training, while maintaining stability and theoretical guarantees.

Guilherme Ramos, A. Pedro Aguiar, Soummya Kar et al.

Average consensus protocols emerge with a central role in distributed systems and decision-making such as distributed information fusion, distributed optimization, distributed estimation, and control. A key advantage of these protocols is that agents exchange and reveal their state information only to their neighbors. Yet, it can raise privacy concerns in situations where the agents' states contain sensitive information. In this paper, we propose a novel (noiseless) privacy preserving distributed algorithms for multi-agent systems to reach an average consensus. The main idea of the algorithms is that each agent runs a (small) network with a crafted structure and dynamics to form a network of networks (i.e., the connection between the newly created networks and their interconnections respecting the initial network connections). Together with a re-weighting of the dynamic parameters dictating the inter-agent dynamics and the initial states, we show that it is possible to ensure that the value of each node converges to the consensus value of the original network. Furthermore, we show that, under mild assumptions, it is possible to craft the dynamics such that the design can be achieved in a distributed fashion. Finally, we illustrate the proposed algorithm with examples.

Orlando Romero, Subhro Das, Pin-Yu Chen et al.

Out of the recent advances in systems and control (S\&C)-based analysis of optimization algorithms, not enough work has been specifically dedicated to machine learning (ML) algorithms and its applications. This paper addresses this gap by illustrating how (discrete-time) Lyapunov stability theory can serve as a powerful tool to aid, or even lead, in the analysis (and potential design) of optimization algorithms that are not necessarily gradient-based. The particular ML problem that this paper focuses on is that of parameter estimation in an incomplete-data Bayesian framework via the popular optimization algorithm known as maximum a posteriori expectation-maximization (MAP-EM). Following first principles from dynamical systems stability theory, conditions for convergence of MAP-EM are developed. Furthermore, if additional assumptions are met, we show that fast convergence (linear or quadratic) is achieved, which could have been difficult to unveil without our adopted S\&C approach. The convergence guarantees in this paper effectively expand the set of sufficient conditions for EM applications, thereby demonstrating the potential of similar S\&C-based convergence analysis of other ML algorithms.

Matilde Tristany Farinha, Sérgio Pequito, Pedro A. Santos et al.

Artificial neural networks, one of the most successful approaches to supervised learning, were originally inspired by their biological counterparts. However, the most successful learning algorithm for artificial neural networks, backpropagation, is considered biologically implausible. We contribute to the topic of biologically plausible neuronal learning by building upon and extending the equilibrium propagation learning framework. Specifically, we introduce: a new neuronal dynamics and learning rule for arbitrary network architectures; a sparsity-inducing method able to prune irrelevant connections; a dynamical-systems characterization of the models, using Lyapunov theory.

Sarthak Chatterjee, Orlando Romero, Sérgio Pequito

The Expectation-Maximization (EM) algorithm is one of the most popular methods used to solve the problem of parametric distribution-based clustering in unsupervised learning. In this paper, we propose to analyze a generalized EM (GEM) algorithm in the context of Gaussian mixture models, where the maximization step in the EM is replaced by an increasing step. We show that this GEM algorithm can be understood as a linear time-invariant (LTI) system with a feedback nonlinearity. Therefore, we explore some of its convergence properties by leveraging tools from robust control theory. Lastly, we explain how the proposed GEM can be designed, and present a pedagogical example to understand the advantages of the proposed approach.

Gaurav Gupta, Sergio Pequito, Paul Bogdan

Despite significant effort in understanding complex systems (CS), we lack a theory for modeling, inference, analysis and efficient control of time-varying complex networks (TVCNs) in uncertain environments. From brain activity dynamics to microbiome, and even chromatin interactions within the genome architecture, many such TVCNs exhibits a pronounced spatio-temporal fractality. Moreover, for many TVCNs only limited information (e.g., few variables) is accessible for modeling, which hampers the capabilities of analytical tools to uncover the true degrees of freedom and infer the CS model, the hidden states and their parameters. Another fundamental limitation is that of understanding and unveiling of unknown drivers of the dynamics that could sporadically excite the network in ways that straightforward modeling does not work due to our inability to model non-stationary processes. Towards addressing these challenges, in this paper, we consider the problem of learning the fractional dynamical complex networks under unknown unknowns (i.e., hidden drivers) and partial observability (i.e., only partial data is available). More precisely, we consider a generalized modeling approach of TVCNs consisting of discrete-time fractional dynamical equations and propose an iterative framework to determine the network parameterization and predict the state of the system. We showcase the performance of the proposed framework in the context of task classification using real electroencephalogram data.

Orlando Romero, Sarthak Chatterjee, Sérgio Pequito

In this paper, we propose a dynamical systems perspective of the Expectation-Maximization (EM) algorithm. More precisely, we can analyze the EM algorithm as a nonlinear state-space dynamical system. The EM algorithm is widely adopted for data clustering and density estimation in statistics, control systems, and machine learning. This algorithm belongs to a large class of iterative algorithms known as proximal point methods. In particular, we re-interpret limit points of the EM algorithm and other local maximizers of the likelihood function it seeks to optimize as equilibria in its dynamical system representation. Furthermore, we propose to assess its convergence as asymptotic stability in the sense of Lyapunov. As a consequence, we proceed by leveraging recent results regarding discrete-time Lyapunov stability theory in order to establish asymptotic stability (and thus, convergence) in the dynamical system representation of the EM algorithm.

Sarthak Chatterjee, Sérgio Pequito

Fractional-order dynamical systems are used to describe processes that exhibit long-term memory with power-law dependence. Notable examples include complex neurophysiological signals such as electroencephalogram (EEG) and blood-oxygen-level dependent (BOLD) signals. When analyzing different neurophysiological signals and other signals with different origin (for example, biological systems), we often find the presence of artifacts, that is, recorded activity that is due to external causes and does not have its origins in the system of interest. In this paper, we consider the problem of estimating the states of a discrete-time fractional-order dynamical system when there are artifacts present in some of the sensor measurements. Specifically, we provide necessary and sufficient conditions that ensure we can retrieve the system states even in the presence of artifacts. We provide a state estimation algorithm that can estimate the states of the system in the presence of artifacts. Finally, we present illustrative examples of our main results using real EEG data.

Jingqi Li, Ximing Chen, Sérgio Pequito et al.

In this paper, we study the target controllability problem of networked dynamical systems, in which we are tasked to steer a subset of network states towards a desired objective. More specifically, we derive necessary and sufficient conditions for the structural target controllability problem of linear time-invariant (LTI) systems with symmetric state matrices, such as undirected dynamical networks with unknown link weights. To achieve our goal, we first characterize the generic rank of symmetrically structured matrices, as well as the modes of any numerical realization. Subsequently, we provide a graph-theoretic necessary and sufficient condition for the structural controllability of undirected networks with multiple control nodes. Finally, we derive a graph-theoretic necessary and sufficient condition for structural target controllability of undirected networks. Remarkably, apart from the standard reachability condition, only local topological information is needed for the verification of structural target controllability.

Gaurav Gupta, Sergio Pequito, Paul Bogdan

We consider the problem of maximizing non-negative non-decreasing set functions. Although most of the recent work focus on exploiting submodularity, it turns out that several objectives we encounter in practice are not submodular. Nonetheless, often we leverage the greedy algorithms used in submodular functions to determine a solution to the non-submodular functions. Hereafter, we propose to address the original problem by \emph{approximating} the non-submodular function and analyze the incurred error, as well as the performance trade-offs. To quantify the approximation error, we introduce a novel concept of $δ$-approximation of a function, which we used to define the space of submodular functions that lie within an approximation error. We provide necessary conditions on the existence of such $δ$-approximation functions, which might not be unique. Consequently, we characterize this subspace which we refer to as \emph{region of submodularity}. Furthermore, submodular functions are known to lead to different sub-optimality guarantees, so we generalize those dependencies upon a $δ$-approximation into the notion of \emph{greedy curvature}. Finally, we used this latter notion to simplify some of the existing results and efficiently (i.e., linear complexity) determine tightened bounds on the sub-optimality guarantees using objective functions commonly used in practical setups and validate them using real data.

Gaurav Gupta, Sergio Pequito, Paul Bogdan

Brain interfaces are cyber-physical systems that aim to harvest information from the (physical) brain through sensing mechanisms, extract information about the underlying processes, and decide/actuate accordingly. Nonetheless, the brain interfaces are still in their infancy, but reaching to their maturity quickly as several initiatives are released to push forward their development (e.g., NeuraLink by Elon Musk and `typing-by-brain' by Facebook). This has motivated us to revisit the design of EEG-based non-invasive brain interfaces. Specifically, current methodologies entail a highly skilled neuro-functional approach and evidence-based \emph{a priori} knowledge about specific signal features and their interpretation from a neuro-physiological point of view. Hereafter, we propose to demystify such approaches, as we propose to leverage new time-varying complex network models that equip us with a fractal dynamical characterization of the underlying processes. Subsequently, the parameters of the proposed complex network models can be explained from a system's perspective, and, consecutively, used for classification using machine learning algorithms and/or actuation laws determined using control system's theory. Besides, the proposed system identification methods and techniques have computational complexities comparable with those currently used in EEG-based brain interfaces, which enable comparable online performances. Furthermore, we foresee that the proposed models and approaches are also valid using other invasive and non-invasive technologies. Finally, we illustrate and experimentally evaluate this approach on real EEG-datasets to assess and validate the proposed methodology. The classification accuracies are high even on having less number of training samples.

Gaurav Gupta, Sergio Pequito, Paul Bogdan

This paper focuses on analysis and design of time-varying complex networks having fractional order dynamics. These systems are key in modeling the complex dynamical processes arising in several natural and man made systems. Notably, examples include neurophysiological signals such as electroencephalogram (EEG) that captures the variation in potential fields, and blood oxygenation level dependent (BOLD) signal, which serves as a proxy for neuronal activity. Notwithstanding, the complex networks originated by locally measuring EEG and BOLD are often treated as isolated networks and do not capture the dependency from external stimuli, e.g., originated in subcortical structures such as the thalamus and the brain stem. Therefore, we propose a paradigm-shift towards the analysis of such complex networks under unknown unknowns (i.e., excitations). Consequently, the main contributions of the present paper are threefold: (i) we present an alternating scheme that enables to determine the best estimate of the model parameters and unknown stimuli; (ii) we provide necessary and sufficient conditions to ensure that it is possible to retrieve the state and unknown stimuli; and (iii) upon these conditions we determine a small subset of variables that need to be measured to ensure that both state and input can be recovered, while establishing sub-optimality guarantees with respect to the smallest possible subset. Finally, we present several pedagogical examples of the main results using real data collected from an EEG wearable device.

J. Frederico Carvalho, Sergio Pequito, A. Pedro Aguiar et al.

In this paper, for linear time-invariant plants, where a collection of possible inputs and outputs are known a priori, we address the problem of determining the communication between outputs and inputs, i.e., information patterns, such that desired control objectives of the closed-loop system (for instance, stabilizability) through static output feedback may be ensured. We address this problem in the structural system theoretic context. To this end, given a specified structural pattern (locations of zeros/non-zeros) of the plant matrices, we introduce the concept of essential information patterns, i.e., communication patterns between outputs and inputs that satisfy the following conditions: (i) ensure arbitrary spectrum assignment of the closed-loop system, using static output feedback constrained to the information pattern, for almost all possible plant instances with the specified structural pattern; and (ii) any communication failure precludes the resulting information pattern from attaining the pole placement objective in (i). Subsequently, we study the problem of determining essential information patterns. First, we provide several necessary and sufficient conditions to verify whether a specified information pattern is essential or not. Further, we show that such conditions can be verified by resorting to algorithms with polynomial complexity (in the dimensions of the state, input and output). Although such verification can be performed efficiently, it is shown that the problem of determining essential information patterns is in general NP-hard. The main results of the paper are illustrated through examples.

Joao Carvalho, Sergio Pequito, A. Pedro Aguiar et al.

Motivated by the development and deployment of large-scale dynamical systems, often composed of geographically distributed smaller subsystems, we address the problem of verifying their controllability in a distributed manner. In this work we study controllability in the structural system theoretic sense, structural controllability. In other words, instead of focusing on a specific numerical system realization, we provide guarantees for equivalence classes of linear time-invariant systems on the basis of their structural sparsity patterns, i.e., location of zero/nonzero entries in the plant matrices. To this end, we first propose several necessary and/or sufficient conditions to ensure structural controllability of the overall system, on the basis of the structural patterns of the subsystems and their interconnections. The proposed verification criteria are shown to be efficiently implementable (i.e., with polynomial time complexity in the number of the state variables and inputs) in two important subclasses of interconnected dynamical systems: similar (i.e., every subsystem has the same structure), and serial (i.e., every subsystem outputs to at most one other subsystem). Secondly, we provide a distributed algorithm to verify structural controllability for interconnected dynamical systems. The proposed distributed algorithm is efficient and implementable at the subsystem level; the algorithm is iterative, based on communication among (physically) interconnected subsystems, and requires only local model and interconnection knowledge at each subsystem.