Karl Henrik Johansson

Mahmoud Selim, Amr Alanwar, Shreyas Kousik et al. · gatech

Reinforcement learning (RL) is capable of sophisticated motion planning and control for robots in uncertain environments. However, state-of-the-art deep RL approaches typically lack safety guarantees, especially when the robot and environment models are unknown. To justify widespread deployment, robots must respect safety constraints without sacrificing performance. Thus, we propose a Black-box Reachability-based Safety Layer (BRSL) with three main components: (1) data-driven reachability analysis for a black-box robot model, (2) a trajectory rollout planner that predicts future actions and observations using an ensemble of neural networks trained online, and (3) a differentiable polytope collision check between the reachable set and obstacles that enables correcting unsafe actions. In simulation, BRSL outperforms other state-of-the-art safe RL methods on a Turtlebot 3, a quadrotor, a trajectory-tracking point mass, and a hexarotor in wind with an unsafe set adjacent to the area of highest reward.

Bart Besselink, Karl H. Johansson

A novel delay-based spacing policy for the control of vehicle platoons is introduced together with a notion of disturbance string stability. The delay-based spacing policy specifies the desired inter-vehicular distance between vehicles and guarantees that all vehicles track the same spatially varying reference velocity profile, as is for example required for heavy-duty vehicles driving over hilly terrain. Disturbance string stability is a notion of string stability of vehicle platoons subject to external disturbances on all vehicles that guarantees that perturbations do not grow unbounded as they propagate through the platoon. Specifically, a control design approach in the spatial domain is presented that achieves tracking of the desired spacing policy and guarantees disturbance string stability with respect to a spatially varying reference velocity. The results are illustrated by means of simulations.

Julien M. Hendrickx, Karl Henrik Johansson, Raphael M. Jungers et al.

The resilience of Supervisory Control and Data Acquisition (SCADA) systems for electric power networks for certain cyber-attacks is considered. We analyze the vulnerability of the measurement system to false data attack on communicated measurements. The vulnerability analysis problem is shown to be NP-hard, meaning that unless $P = NP$ there is no polynomial time algorithm to analyze the vulnerability of the system. Nevertheless, we identify situations, such as the full measurement case, where it can be solved efficiently. In such cases, we show indeed that the problem can be cast as a generalization of the minimum cut problem involving costly nodes. We further show that it can be reformulated as a standard minimum cut problem (without costly nodes) on a modified graph of proportional size. An important consequence of this result is that our approach provides the first exact efficient algorithm for the vulnerability analysis problem under the full measurement assumption. Furthermore, our approach also provides an efficient heuristic algorithm for the general NP-hard problem. Our results are illustrated by numerical studies on benchmark systems including the IEEE 118-bus system.

Kin Cheong Sou, Henrik Sandberg, Karl Henrik Johansson

This paper considers a smart grid cyber-security problem analyzing the vulnerabilities of electric power networks to false data attacks. The analysis problem is related to a constrained cardinality minimization problem. The main result shows that an $l_1$ relaxation technique provides an exact optimal solution to this cardinality minimization problem. The proposed result is based on a polyhedral combinatorics argument. It is different from well-known results based on mutual coherence and restricted isometry property. The results are illustrated on benchmarks including the IEEE 118-bus and 300-bus systems.

Sebastian van de Hoef, Karl H. Johansson, Dimos V. Dimarogonas

The problem of how to coordinate a large fleet of trucks with given itinerary to enable fuel-efficient platooning is considered. Platooning is a promising technology that enables trucks to save significant amounts of fuel by driving close together and thus reducing air drag. A setting is considered in which each truck in a fleet is provided with a start location, a destination, a departure time, and an arrival deadline from a higher planning level. Fuel-efficient plans should be computed. The plans consist of routes and speed profiles that allow trucks to arrive by their arrival deadlines. Hereby, trucks can meet on common parts of their routes and form platoons, resulting in decreased fuel consumption. We formulate a combinatorial optimization problem that combines plans involving only two vehicles. We show that this problem is hard to solve for large problem instances. Hence a heuristic algorithm is proposed. The resulting plans are further optimized using convex optimization techniques. The method is evaluated with Monte Carlo simulations in a realistic setting. We demonstrate that the proposed algorithm can compute plans for thousands of trucks and that significant fuel savings can be achieved.

Farhad Farokhi, Iman Shames, Karl H. Johansson

A conventional way to handle model predictive control (MPC) problems distributedly is to solve them via dual decomposition and gradient ascent. However, at each time-step, it might not be feasible to wait for the dual algorithm to converge. As a result, the algorithm might be needed to be terminated prematurely. One is then interested to see if the solution at the point of termination is close to the optimal solution and when one should terminate the algorithm if a certain distance to optimality is to be guaranteed. In this chapter, we look at this problem for distributed systems under general dynamical and performance couplings, then, we make a statement on validity of similar results where the problem is solved using alternating direction method of multipliers.

Julien M. Hendrickx, Guodong Shi, Karl H. Johansson

We discuss the possibility of reaching consensus in finite time using only linear iterations, with the additional restrictions that the update matrices must be stochastic with positive diagonals and consistent with a given graph structure. We show that finite-time average consensus can always be achieved for connected undirected graphs. For directed graphs, we show some necessary conditions for finite-time consensus, including strong connectivity and the presence of a simple cycle of even length.

Xiaoqiang Ren, Junfeng Wu, Karl H. Johansson et al.

Jointly optimal transmission power control and remote estimation over an infinite horizon is studied. A sensor observes a dynamic process and sends its observations to a remote estimator over a wireless fading channel characterized by a time-homogeneous Markov chain. The successful transmission probability depends on both the channel gains and the transmission power used by the sensor. The transmission power control rule and the remote estimator should be jointly designed, aiming to minimize an infinite-horizon cost consisting of the power usage and the remote estimation error. A first question one may ask is: Does this joint optimization problem have a solution? We formulate the joint optimization problem as an average cost belief-state Markov decision process and answer the question by proving that there exists an optimal deterministic and stationary policy. We then show that when the monitored dynamic process is scalar, the optimal remote estimates depend only on the most recently received sensor observation, and the optimal transmission power is symmetric and monotonically increasing with respect to the innovation error.

Guodong Shi, Mikael Johansson, Karl Henrik Johansson

The dynamics of an agreement protocol interacting with a disagreement process over a common random network is considered. The model can represent the spreading of true and false information over a communication network, the propagation of faults in a large-scale control system, or the development of trust and mistrust in a society. At each time instance and with a given probability, a pair of network nodes are selected to interact. At random each of the nodes then updates its state towards the state of the other node (attraction), away from the other node (repulsion), or sticks to its current state (neglect). Agreement convergence and disagreement divergence results are obtained for various strengths of the updates for both symmetric and asymmetric update rules. Impossibility theorems show that a specific level of attraction is required for almost sure asymptotic agreement and a specific level of repulsion is required for almost sure asymptotic disagreement. A series of sufficient and/or necessary conditions are then established for agreement convergence or disagreement divergence. In particular, under symmetric updates, a critical convergence measure in the attraction and repulsion update strength is found, in the sense that the asymptotic property of the network state evolution transits from agreement convergence to disagreement divergence when this measure goes from negative to positive. The result can be interpreted as a tight bound on how much bad action needs to be injected in a dynamic network in order to consistently steer its overall behavior away from consensus.

Shuang Wu, Xiaoqiang Ren, Qing-Shan Jia et al.

We consider optimal sensor scheduling with unknown communication channel statistics. We formulate two types of scheduling problems with the communication rate being a soft or hard constraint, respectively. We first present some structural results on the optimal scheduling policy using dynamic programming and assuming the channel statistics is known. We prove that the Q-factor is monotonic and submodular, which leads to the threshold-like structures in both types of problems. Then we develop a stochastic approximation and parameter learning frameworks to deal with the two scheduling problems with unknown channel statistics. We utilize their structures to design specialized learning algorithms. We prove the convergence of these algorithms. Performance improvement compared with the standard Q-learning algorithm is shown through numerical examples.

Kun Liu, Emilia Fridman, Karl Henrik Johansson et al.

The Jensen inequality has been recognized as a powerful tool to deal with the stability of time-delay systems. Recently, a new inequality that encompasses the Jensen inequality was proposed for the stability analysis of systems with finite delays. In this paper, we first present a generalized integral inequality and its double integral extension. It is shown how these inequalities can be applied to improve the stability result for linear continuous-time systems with gamma-distributed delays. Then, for the discrete-time counterpart we provide an extended Jensen summation inequality with infinite sequences, which leads to less conservative stability conditions for linear discrete-time systems with poisson-distributed delays. The improvements obtained thanks to the introduced generalized inequalities are demonstrated by examples.

Yulong Gao, Karl H. Johansson, Lihua Xie

This paper investigates stochastic invariance for control systems through probabilistic controlled invariant sets (PCISs). As a natural complement to robust controlled invariant sets~(RCISs), we propose finite- and infinite-horizon PCISs, and explore their relation to RICSs. We design iterative algorithms to compute the PCIS within a given set. For systems with discrete spaces, the computations of the finite- and infinite-horizon PCISs at each iteration are based on linear programming and mixed integer linear programming, respectively. The algorithms are computationally tractable and terminate in a finite number of steps. For systems with continuous spaces, we show how to discretize the spaces and prove the convergence of the approximation when computing the finite-horizon PCISs. In addition, it is shown that an infinite-horizon PCIS can be computed by the stochastic backward reachable set from the RCIS contained in it. These PCIS algorithms are applicable to practical control systems. Simulations are given to illustrate the effectiveness of the theoretical results for motion planning.

Muhammad Umar B. Niazi, John Cao, Xudong Sun et al.

Designing Luenberger observers for nonlinear systems involves the challenging task of transforming the state to an alternate coordinate system, possibly of higher dimensions, where the system is asymptotically stable and linear up to output injection. The observer then estimates the system's state in the original coordinates by inverting the transformation map. However, finding a suitable injective transformation whose inverse can be derived remains a primary challenge for general nonlinear systems. We propose a novel approach that uses supervised physics-informed neural networks to approximate both the transformation and its inverse. Our method exhibits superior generalization capabilities to contemporary methods and demonstrates robustness to both neural network's approximation errors and system uncertainties.

Junfeng Wu, Ling Shi, Lihua Xie et al.

In this paper, we consider the peak-covariance stability of Kalman filtering subject to packet losses. The length of consecutive packet losses is governed by a time-homogeneous finite-state Markov chain. We establish a sufficient condition for peak-covariance stability and show that this stability check can be recast as a linear matrix inequality (LMI) feasibility problem. Comparing with the literature, the stability condition given in this paper is invariant with respect to similarity state transformations; moreover, our condition is proved to be less conservative than the existing results. Numerical examples are provided to demonstrate the effectiveness of our result.

Xiaoqiang Ren, Yilin Mo, Jie Chen et al.

This paper studies static state estimation in multi-sensor settings, with a caveat that an unknown subset of the sensors are compromised by an adversary, whose measurements can be manipulated arbitrarily. The attacker is able to compromise $q$ out of $m$ sensors. A new performance metric, which quantifies the asymptotic decay rate for the probability of having an estimation error larger than $δ$, is proposed. We develop an optimal estimator for the new performance metric with a fixed $δ$, which is the Chebyshev center of a union of ellipsoids. We further provide an estimator that is optimal for every $δ$, for the special case where the sensors are homogeneous. Numerical examples are given to elaborate the results.

Maben Rabi, Chithrupa Ramesh, Karl Henrik Johansson

For a networked control system, we consider the problem of encoder and controller design. We study a discrete-time linear plant with a finite horizon performance cost, comprising of a quadratic function of the states and controls, and an additive communication cost. We study separation in design of the encoder and controller, along with related closed-loop properties such as the dual effect and certainty equivalence. We consider three basic formats for encoder outputs: quantized samples, real-valued samples at event-triggered times, and real-valued samples over additive noise channels. If the controller and encoder are dynamic, then we show that the performance cost is minimized by a separated design: the controls are updated at each time instant as per a certainty equivalence law, and the encoder is chosen to minimize an aggregate quadratic distortion of the estimation error. This separation is shown to hold even though a dual effect is present in the closed-loop system. We also show that this separated design need not be optimal when the controller or encoder are to be chosen from within restricted classes.

Milos S. Stankovic, Srdjan S. Stankovic, Karl Henrik Johansson

In this paper a new distributed asynchronous algorithm is proposed for time synchronization in networks with random communication delays, measurement noise and communication dropouts. Three different types of the drift correction algorithm are introduced, based on different kinds of local time increments. Under nonrestrictive conditions concerning network properties, it is proved that all the algorithm types provide convergence in the mean square sense and with probability one (w.p.1) of the corrected drifts of all the nodes to the same value (consensus). An estimate of the convergence rate of these algorithms is derived. For offset correction, a new algorithm is proposed containing a compensation parameter coping with the influence of random delays and special terms taking care of the influence of both linearly increasing time and drift correction. It is proved that the corrected offsets of all the nodes converge in the mean square sense and w.p.1. An efficient offset correction algorithm based on consensus on local compensation parameters is also proposed. It is shown that the overall time synchronization algorithm can also be implemented as a flooding algorithm with one reference node. It is proved that it is possible to achieve bounded error between local corrected clocks in the mean square sense and w.p.1. Simulation results provide an additional practical insight into the algorithm properties and show its advantage over the existing methods.

Sebastian van de Hoef, Karl H. Johansson, Dimos V. Dimarogonas

The problem of maximizing the probability of two trucks being coordinated to merge into a platoon on a highway is considered. Truck platooning is a promising technology that allows heavy vehicles to save fuel by driving with small automatically controlled inter-vehicle distances. In order to leverage the full potential of platooning, platoons can be formed dynamically en route by small adjustments to their speeds. However, in heavily used parts of the road network, travel times are subject to random disturbances originating from traffic, weather and other sources. We formulate this problem as a stochastic dynamic programming problem over a finite horizon, for which solutions can be computed using a backwards recursion. By exploiting the characteristics of the problem, we derive bounds on the set of states that have to be explored at every stage, which in turn reduces the complexity of computing the solution. Simulations suggest that the approach is applicable to realistic problem instances.

Sebastian van de Hoef, Karl H. Johansson, Dimos V. Dimarogonas

Vehicle platooning facilitates the partial automation of vehicles and can significantly reduce fuel consumption. Mobile communication infrastructure makes it possible to dynamically coordinate the formation of platoons en route. We consider a centralized system that provides trucks with routes and speed profiles allowing them to dynamically form platoons during their journeys. For this to work, all possible pairs of vehicles that can platoon based on their location, destination, and other constraints have to be identified. The presented approach scales well to large vehicle fleets and realistic road networks by extracting features from the transport assignments of the vehicles and rules out a majority of possible pairs based on these features only. Merely a small number of remaining pairs are considered in depth by a complete and computationally expensive algorithm. This algorithm conclusively decides if platooning is possible for a pair based on the complete data associated with the two vehicles. We derive appropriate features for the problem and demonstrate the effectiveness of the approach in a simulation example.

Xingkang He, Xiaoqiang Ren, Henrik Sandberg et al.

In this paper, we consider a secure distributed filtering problem for linear time-invariant systems with bounded noises and unstable dynamics under compromised observations. A malicious attacker is able to compromise a subset of the agents and manipulate the observations arbitrarily. We first propose a recursive distributed filter consisting of two parts at each time. The first part employs a saturation-like scheme, which gives a small gain if the innovation is too large. The second part is a consensus operation of state estimates among neighboring agents. A sufficient condition is then established for the boundedness of estimation error, which is with respect to network topology, system structure, and the maximal compromised agent subset. We further provide an equivalent statement, which connects to 2s-sparse observability in the centralized framework in certain scenarios, such that the sufficient condition is feasible. Numerical simulations are finally provided to illustrate the developed results.

Moritz Wiese, Karl Henrik Johansson, Tobias J. Oechtering et al.

Uncertain wiretap channels are introduced. Their zero-error secrecy capacity is defined. If the sensor-estimator channel is perfect, it is also calculated. Further properties are discussed. The problem of estimating a dynamical system with nonstochastic disturbances is studied where the sensor is connected to the estimator and an eavesdropper via an uncertain wiretap channel. The estimator should obtain a uniformly bounded estimation error whereas the eavesdropper's error should tend to infinity. It is proved that the system can be estimated securely if the zero-error capacity of the sensor-estimator channel is strictly larger than the logarithm of the system's unstable pole and the zero-error secrecy capacity of the uncertain wiretap channel is positive.

Bart Besselink, Karl H. Johansson, Arjan van der Schaft

This paper introduces assume/guarantee contracts on continuous-time control systems, hereby extending contract theories for discrete systems to certain new model classes and specifications. Contracts are regarded as formal characterizations of control specifications, providing an alternative to specifications in terms of dissipativity properties or set-invariance. The framework has the potential to capture a richer class of specifications more suitable for complex engineering systems. The proposed contracts are supported by results that enable the verification of contract implementation and the comparison of contracts. These results are illustrated by an example of a vehicle following system.

Riccardo Sven Risuleo, Marco Molinari, Giulio Bottegal et al.

This paper describes a benchmark consisting of a set of synthetic measurements relative to an office environment simulated with the software IDA-ICE. The simulated environment reproduces a laboratory at the KTH-EES Smart Building, equipped with a building management system. The data set contains records collected over a period of several days. The signals to CO$_2$ concentration, mechanical ventilation airflows, air infiltrations and occupancy. Information on door and window opening is also available. This benchmark is intended for testing data-based modeling techniques. The ultimate goal is the development of models to improve the forecast and control of environmental variables. Among the numerous challenges related to this framework, we point out the problem of occupancy estimation using information on CO$_2$ concentration. This can be seen as a blind identification problem. For benchmarking purposes, we present two different identification approaches: a baseline overparametrization method and a kernel-based method.

Farhad Farokhi, Karl H. Johansson

The value of plant model information available in the control design process is discussed. We design optimal state-feedback controllers for interconnected discrete-time linear systems with stochastically-varying parameters. The parameters are assumed to be independently and identically distributed random variables in time. The design of each controller relies only on (i) exact local plant model information and (ii) statistical beliefs about the model of the rest of the system. We consider both finite-horizon and infinite-horizon quadratic cost functions. The optimal state-feedback controller is derived in both cases. The optimal controller is shown to be linear in the state and to depend on the model parameters and their statistics in a particular way. Furthermore, we study the value of model information in optimal control design using the performance degradation ratio which is defined as the supremum (over all possible initial conditions) of the ratio of the cost of the optimal controller with limited model information scaled by the cost of the optimal controller with full model information. An upper bound for the performance degradation ratio is presented for the case of fully-actuated subsystems. Comparisons are made between designs based on limited, statistical, and full model information. Throughout the paper, we use a power network example to illustrate concepts and results.

Dominik Baumann, Friedrich Solowjow, Karl Henrik Johansson et al.

In networked control systems, communication is a shared and therefore scarce resource. Event-triggered control (ETC) can achieve high performance control with a significantly reduced amount of samples compared to classical, periodic control schemes. However, ETC methods usually rely on the availability of an accurate dynamics model, which is oftentimes not readily available. In this paper, we propose a novel event-triggered pulse control strategy that learns dynamics models if necessary. In addition to adapting to changing dynamics, the method also represents a suitable replacement for the integral part typically used in periodic control.

Matin Jafarian, Xinlei Yi, Mohammad Pirani et al.

This paper studies the synchronization of a finite number of Kuramoto oscillators in a frequency-dependent bidirectional tree network. We assume that the coupling strength of each link in each direction is equal to the product of a common coefficient and the exogenous frequency of its corresponding head oscillator. We derive a sufficient condition for the common coupling strength in order to guarantee frequency synchronization in tree networks. Moreover, we discuss the dependency of the obtained bound on both the graph structure and the way that exogenous frequencies are distributed. Further, we present an application of the obtained result by means of an event-triggered algorithm for achieving frequency synchronization in a star network assuming that the common coupling coefficient is given.

Matin Jafarian, Henrik Sandberg, Karl H. Johansson

This paper studies the stability of voltage dynamics for a power network in which nodal voltages are controlled by means of quadratic droop controllers with nonlinear AC reactive power as inputs. We show that the voltage dynamics is a Lotka-Volterra system, which is a class of nonlinear positive systems. We study the stability of the closed-loop system by proving a uniform ultimate boundedness result and investigating conditions under which the network is cooperative. We then restrict to study the stability of voltage dynamics under a decoupling assumption (i.e., zero relative angles). We analyze the existence and uniqueness of the equilibrium in the interior of the positive orthant for the system and prove an asymptotic stability result.

F. Farokhi, C. Langbort, K. H. Johansson

The design of optimal disturbance accommodation and servomechanism controllers with limited plant model information is considered in this paper. Their closed-loop performance are compared using a performance metric called competitive ratio which is the worst-case ratio of the cost of a given control design strategy to the cost of the optimal control design with full model information. It was recently shown that when it comes to designing optimal centralized or partially structured decentralized state-feedback controllers with limited model information, the best control design strategy in terms of competitive ratio is a static one. This is true even though the optimal structured decentralized state-feedback controller with full model information is dynamic. In this paper, we show that, in contrast, the best limited model information control design strategy for the disturbance accommodation problem gives a dynamic controller. We find an explicit minimizer of the competitive ratio and we show that it is undominated, that is, there is no other control design strategy that performs better for all possible plants while having the same worst-case ratio. This optimal controller can be separated into a static feedback law and a dynamic disturbance observer. For constant disturbances, it is shown that this structure corresponds to proportional-integral control.

Mahmoud Selim, Amr Alanwar, M. Watheq El-Kharashi et al.

Reinforcement learning (RL) algorithms can achieve state-of-the-art performance in decision-making and continuous control tasks. However, applying RL algorithms on safety-critical systems still needs to be well justified due to the exploration nature of many RL algorithms, especially when the model of the robot and the environment are unknown. To address this challenge, we propose a data-driven safety layer that acts as a filter for unsafe actions. The safety layer uses a data-driven predictive controller to enforce safety guarantees for RL policies during training and after deployment. The RL agent proposes an action that is verified by computing the data-driven reachability analysis. If there is an intersection between the reachable set of the robot using the proposed action, we call the data-driven predictive controller to find the closest safe action to the proposed unsafe action. The safety layer penalizes the RL agent if the proposed action is unsafe and replaces it with the closest safe one. In the simulation, we show that our method outperforms state-of-the-art safe RL methods on the robotics navigation problem for a Turtlebot 3 in Gazebo and a quadrotor in Unreal Engine 4 (UE4).

Zifan Wang, Yi Shen, Zachary I. Bell et al.

We consider risk-averse learning in repeated unknown games where the goal of the agents is to minimize their individual risk of incurring significantly high cost. Specifically, the agents use the conditional value at risk (CVaR) as a risk measure and rely on bandit feedback in the form of the cost values of the selected actions at every episode to estimate their CVaR values and update their actions. A major challenge in using bandit feedback to estimate CVaR is that the agents can only access their own cost values, which, however, depend on the actions of all agents. To address this challenge, we propose a new risk-averse learning algorithm with momentum that utilizes the full historical information on the cost values. We show that this algorithm achieves sub-linear regret and matches the best known algorithms in the literature. We provide numerical experiments for a Cournot game that show that our method outperforms existing methods.

F. Farokhi, C. Langbort, K. H. Johansson

The design of optimal dynamic disturbance accommodation controller with limited model information is considered. We adapt the family of limited model information control design strategies, defined earlier by the authors, to handle dynamic controllers. This family of limited model information design strategies construct subcontrollers distributively by accessing only local plant model information. The closed-loop performance of the dynamic controllers that they can produce are studied using a performance metric called the competitive ratio which is the worst case ratio of the cost a control design strategy to the cost of the optimal control design with full model information.

Phoebus Chen, Chithrupa Ramesh, Karl H. Johansson

Much of the current theory of networked control systems uses simple point-to-point communication models as an abstraction of the underlying network. As a result, the controller has very limited information on the network conditions and performs suboptimally. This work models the underlying wireless multihop mesh network as a graph of links with transmission success probabilities, and uses a recursive Bayesian estimator to provide packet delivery predictions to the controller. The predictions are a joint probability distribution on future packet delivery sequences, and thus capture correlations between successive packet deliveries. We look at finite horizon LQG control over a lossy actuation channel and a perfect sensing channel, both without delay, to study how the controller can compensate for predicted network outages.

Nicola Bastianello, Apostolos I. Rikos, Karl H. Johansson

In this paper we consider online distributed learning problems. Online distributed learning refers to the process of training learning models on distributed data sources. In our setting a set of agents need to cooperatively train a learning model from streaming data. Differently from federated learning, the proposed approach does not rely on a central server but only on peer-to-peer communications among the agents. This approach is often used in scenarios where data cannot be moved to a centralized location due to privacy, security, or cost reasons. In order to overcome the absence of a central server, we propose a distributed algorithm that relies on a quantized, finite-time coordination protocol to aggregate the locally trained models. Furthermore, our algorithm allows for the use of stochastic gradients during local training. Stochastic gradients are computed using a randomly sampled subset of the local training data, which makes the proposed algorithm more efficient and scalable than traditional gradient descent. In our paper, we analyze the performance of the proposed algorithm in terms of the mean distance from the online solution. Finally, we present numerical results for a logistic regression task.

Youcheng Lou, Guodong Shi, Karl Henrik Johansson et al.

In this paper, we propose an approximate projected consensus algorithm for a network to cooperatively compute the intersection of convex sets. Instead of assuming the exact convex projection proposed in the literature, we allow each node to compute an approximate projection and communicate it to its neighbors. The communication graph is directed and time-varying. Nodes update their states by weighted averaging. Projection accuracy conditions are presented for the considered algorithm. They indicate how much projection accuracy is required to ensure global consensus to a point in the intersection set when the communication graph is uniformly jointly strongly connected. We show that $π/4$ is a critical angle error of the projection approximation to ensure a bounded state. A numerical example indicates that this approximate projected consensus algorithm may achieve better performance than the exact projected consensus algorithm in some cases.

Zifan Wang, Yulong Gao, Siyi Wang et al.

Distributional reinforcement learning (DRL) enhances the understanding of the effects of the randomness in the environment by letting agents learn the distribution of a random return, rather than its expected value as in standard RL. At the same time, a main challenge in DRL is that policy evaluation in DRL typically relies on the representation of the return distribution, which needs to be carefully designed. In this paper, we address this challenge for a special class of DRL problems that rely on linear quadratic regulator (LQR) for control, advocating for a new distributional approach to LQR, which we call \emph{distributional LQR}. Specifically, we provide a closed-form expression of the distribution of the random return which, remarkably, is applicable to all exogenous disturbances on the dynamics, as long as they are independent and identically distributed (i.i.d.). While the proposed exact return distribution consists of infinitely many random variables, we show that this distribution can be approximated by a finite number of random variables, and the associated approximation error can be analytically bounded under mild assumptions. Using the approximate return distribution, we propose a zeroth-order policy gradient algorithm for risk-averse LQR using the Conditional Value at Risk (CVaR) as a measure of risk. Numerical experiments are provided to illustrate our theoretical results.

Matin Jafarian, Mohammad H. Mamduhi, Karl H. Johansson

This paper presents the notion of stochastic phase-cohesiveness based on the concept of recurrent Markov chains and studies the conditions under which a discrete-time stochastic Kuramoto model is phase-cohesive. It is assumed that the exogenous frequencies of the oscillators are combined with random variables representing uncertainties. A bidirectional tree network is considered such that each oscillator is coupled to its neighbors with a coupling law which depends on its own noisy exogenous frequency. In addition, an undirected tree network is studied. For both cases, a sufficient condition for the common coupling strength and a necessary condition for the sampling-period are derived such that the stochastic phase-cohesiveness is achieved. The analysis is performed within the stochastic systems framework and validated by means of numerical simulations.

Junfeng Wu, Guodong Shi, Brian D. O. Anderson et al.

In this paper, we investigate probabilistic stability of Kalman filtering over fading channels modeled by $\ast$-mixing random processes, where channel fading is allowed to generate non-stationary packet dropouts with temporal and/or spatial correlations. Upper/lower almost sure (a.s.) stabilities and absolutely upper/lower a.s. stabilities are defined for characterizing the sample-path behaviors of the Kalman filtering. We prove that both upper and lower a.s. stabilities follow a zero-one law, i.e., these stabilities must happen with a probability either zero or one, and when the filtering system is one-step observable, the absolutely upper and lower a.s. stabilities can also be interpreted using a zero-one law. We establish general stability conditions for (absolutely) upper and lower a.s. stabilities. In particular, with one-step observability, we show the equivalence between absolutely a.s. stabilities and a.s. ones, and necessary and sufficient conditions in terms of packet arrival rate are derived; for the so-called non-degenerate systems, we also manage to give a necessary and sufficient condition for upper a.s. stability.

Mehran Zareh, Dimos V. Dimarogonas, Mauro Franceschelli et al.

In this paper consensus in second-order multi-agent systems with a non-periodic sampled-data exchange among agents is investigated. The sampling is random with bounded inter-sampling intervals. It is assumed that each agent has exact knowledge of its own state at any time instant. The considered local interaction rule is PD-type. Sufficient conditions for stability of the consensus protocol to a time-invariant value are derived based on LMIs. Such conditions only require the knowledge of the connectivity of the graph modeling the network topology. Numerical simulations are presented to corroborate the theoretical results.

Farhad Farokhi, Karl H. Johansson

Networked control strategies based on limited information about the plant model usually results in worse closed-loop performance than optimal centralized control with full plant model information. Recently, this fact has been established by utilizing the concept of competitive ratio, which is defined as the worst case ratio of the cost of a control design with limited model information to the cost of the optimal control design with full model information. We show that an adaptive controller, inspired by a controller proposed by Campi and Kumar, with limited plant model information, asymptotically achieves the closed-loop performance of the optimal centralized controller with full model information for almost any plant. Therefore, there exists, at least, one adaptive control design strategy with limited plant model information that can achieve a competitive ratio equal to one. The plant model considered in the paper belongs to a compact set of stochastic linear time-invariant systems and the closed loop performance measure is the ergodic mean of a quadratic function of the state and control input. We illustrate the applicability of the results numerically on a vehicle platooning problem.

Guodong Shi, Alexandre Proutiere, Karl Henrik Johansson

This paper explores the fundamental properties of distributed minimization of a sum of functions with each function only known to one node, and a pre-specified level of node knowledge and computational capacity. We define the optimization information each node receives from its objective function, the neighboring information each node receives from its neighbors, and the computational capacity each node can take advantage of in controlling its state. It is proven that there exist a neighboring information way and a control law that guarantee global optimal consensus if and only if the solution sets of the local objective functions admit a nonempty intersection set for fixed strongly connected graphs. Then we show that for any tolerated error, we can find a control law that guarantees global optimal consensus within this error for fixed, bidirectional, and connected graphs under mild conditions. For time-varying graphs, we show that optimal consensus can always be achieved as long as the graph is uniformly jointly strongly connected and the nonempty intersection condition holds. The results illustrate that nonempty intersection for the local optimal solution sets is a critical condition for successful distributed optimization for a large class of algorithms.

Miguel Aguiar, Amritam Das, Karl H. Johansson

We describe a recurrent neural network (RNN) based architecture to learn the flow function of a causal, time-invariant and continuous-time control system from trajectory data. By restricting the class of control inputs to piecewise constant functions, we show that learning the flow function is equivalent to learning the input-to-state map of a discrete-time dynamical system. This motivates the use of an RNN together with encoder and decoder networks which map the state of the system to the hidden state of the RNN and back. We show that the proposed architecture is able to approximate the flow function by exploiting the system's causality and time-invariance. The output of the learned flow function model can be queried at any time instant. We experimentally validate the proposed method using models of the Van der Pol and FitzHugh Nagumo oscillators. In both cases, the results demonstrate that the architecture is able to closely reproduce the trajectories of these two systems. For the Van der Pol oscillator, we further show that the trained model generalises to the system's response with a prolonged prediction time horizon as well as control inputs outside the training distribution. For the FitzHugh-Nagumo oscillator, we show that the model accurately captures the input-dependent phenomena of excitability.

Zhongqi Sun, Li Dai, Kun Liu et al.

In this paper, two robust model predictive control (MPC) schemes are proposed for tracking control of nonholonomic systems with bounded disturbances: tube-MPC and nominal robust MPC (NRMPC). In tube-MPC, the control signal consists of a control action and a nonlinear feedback law based on the deviation of the actual states from the states of a nominal system. It renders the actual trajectory within a tube centered along the optimal trajectory of the nominal system. Recursive feasibility and input-to-state stability are established and the constraints are ensured by tightening the input domain and the terminal region. While in NRMPC, an optimal control sequence is obtained by solving an optimization problem based on the current state, and the first portion of this sequence is applied to the real system in an open-loop manner during each sampling period. The state of nominal system model is updated by the actual state at each step, which provides additional a feedback. By introducing a robust state constraint and tightening the terminal region, recursive feasibility and input-to-state stability are guaranteed. Simulation results demonstrate the effectiveness of both strategies proposed.

Yu Xing, Xingkang He, Haitao Fang et al.

This paper studies the estimation of network weights for a class of systems with binary-valued observations. In these systems only quantized observations are available for the network estimation. Furthermore, system states are coupled with observations, and the quantization parts are unknown inherent components, which hinder the design of inputs and quantizers. To fulfill the estimation, we propose a recursive algorithm based on stochastic approximation techniques. More precisely, to deal with the temporal dependency of observations and achieve the recursive estimation of network weights, a deterministic objective function is constructed based on the likelihood function by extending the dimension of observations and applying ergodic properties of Markov chains. It is shown that this function is strictly concave and has unique maximum identical to the true parameter vector. Finally, the strong consistency of the algorithm is established. Our recursive algorithm can be applied to online tasks like real-time decision-making and surveillance for networked systems. This work also provides a new scheme for the identification of systems with quantized observations.

Behrad Samari, Henrik Sandberg, Karl H. Johansson et al.

Model order reduction techniques simplify high-dimensional dynamical systems by deriving lower-dimensional models that retain essential system characteristics. These techniques are crucial for the controller design of complex systems while significantly reducing computational costs. Nevertheless, constructing effective reduced-order models (ROMs) poses considerable challenges, particularly for nonlinear dynamical systems. These challenges are further exacerbated when the actual system model is unavailable, a scenario frequently encountered in real-world applications. In this work, we propose a data-driven framework for constructing ROMs of nonlinear dynamical systems with unknown mathematical models, enabling controller synthesis directly from the resulting ROMs. We establish similarity relations between the output trajectories of the original systems and those of their ROMs by employing the notion of simulation functions (SFs), thereby enabling a formal characterization of their closeness. To achieve this, we collect one set of noise-corrupted input-state data from the system during a finite-time experiment, upon which we propose conditions to construct both ROMs and SFs simultaneously. These conditions are formulated as data-dependent semidefinite programs. We demonstrate that the data-driven ROMs obtained can be employed to synthesize controllers for the original unknown systems, ensuring that they satisfy high-level logic specifications. This is accomplished by first designing controllers for the data-driven ROMs and then translating the results back to the original systems via interface functions, designed directly from the proposed data-dependent conditions. We evaluate the efficacy of our data-driven framework through two case studies, including a challenging benchmark from the model reduction literature: a circuit of chained inverter gates with 20 state variables.

Xingkang He, Qian Liu, Junfeng Wu et al.

In this paper, we study a distributed parameter estimation problem with an asynchronous communication protocol over multi-agent systems. Different from traditional time-driven communication schemes, in this work, data can be transmitted between agents intermittently rather than in a steady stream. First, we propose a recursive distributed estimator based on an event-triggered communication scheme, through which each agent can decide whether the current estimate is sent out to its neighbors or not. With this scheme, considerable communications between agents can be effectively reduced. Then, under mild conditions including a collective observability, we provide a design principle of triggering thresholds to guarantee the asymptotic unbiasedness and strong consistency. Furthermore, under certain conditions, we prove that, with probability one, for every agent the time interval between two successive triggered instants goes to infinity as time goes to infinity. Finally, we provide a numerical simulation to validate the theoretical results of this paper.

Farhad Farokhi, Henrik Sandberg, Karl H. Johansson

We present a complexity reduction algorithm for a family of parameter-dependent linear systems when the system parameters belong to a compact semi-algebraic set. This algorithm potentially describes the underlying dynamical system with fewer parameters or state variables. To do so, it minimizes the distance (i.e., H-infinity-norm of the difference) between the original system and its reduced version. We present a sub-optimal solution to this problem using sum-of-squares optimization methods. We present the results for both continuous-time and discrete-time systems. Lastly, we illustrate the applicability of our proposed algorithm on numerical examples.

Takumi Shinohara, Karl Henrik Johansson, Henrik Sandberg

We develop a data-driven framework for assessing the resilience of linear time-invariant systems against malicious false-data-injection sensor attacks. Leveraging sparse observability, we propose data-driven resilience metrics and derive necessary and sufficient conditions for two data-availability scenarios. For attack-free data, we show that when a rank condition holds, the resilience level can be computed exactly from the data alone, without prior knowledge of the system parameters. We then extend the analysis to the case where only poisoned data are available and show that the resulting assessment is necessarily conservative. For both scenarios, we provide algorithms for computing the proposed metrics and show that they can be computed in polynomial time under an additional spectral condition. A numerical example illustrates the efficacy and limitations of the proposed framework.

Elis Stefansson, Karl H. Johansson

We consider optimal planning in a large-scale system formalised as a hierarchical finite state machine (HFSM). A planning algorithm is proposed computing an optimal plan between any two states in the HFSM, consisting of two steps: A pre-processing step that computes optimal exit costs of the machines in the HFSM, with time complexity scaling with the number of machines; and a query step that efficiently computes an optimal plan by removing irrelevant subtrees of the HFSM using the optimal exit costs. The algorithm is reconfigurable in the sense that changes in the HFSM are handled with ease, where the pre-processing step recomputes only the optimal exit costs affected by the change. The algorithm can also exploit compact representations that groups together identical machines in the HFSM, where the algorithm only needs to compute the optimal exit costs for one of the identical machines within each group, thereby avoid unnecessary recomputations. We validate the algorithm on large systems with millions of states and a robotic application. It is shown that our approach outperforms Dijkstra's algorithm, Bidirectional Dijkstra and Contraction Hierarchies.

Mahsa Farjadnia, Katayoun Eshkofti, Albin Apell et al.

This paper investigates non-intrusive occupancy detection methods for residential buildings using environmental sensor data from the KTH Live-In Lab in Stockholm, Sweden. Three machine learning approaches, namely, logistic regression (LR), support vector machines (SVM), and long short-term memory (LSTM) network enhanced with an attention mechanism, are evaluated in terms of predictive performance and computational complexity. The analysis considers the trade-off between sensor availability (investment cost) and prediction accuracy in real applications, as well as the models' cross-apartment generalizability. Hyperparameters for both the SVM and LSTM models are optimized using Bayesian optimization. All three models are evaluated on data collected from apartments not used during training, and on data generated from a calibrated digital model of the testbed. Results show that all models achieve comparable performance on the same-apartment test data (accuracy of approximately 0.83, F1 score of approximately 0.86). When assessed on cross-apartment data, the LSTM model demonstrates the strongest generalization capability (accuracy of 0.84, F1 score of 0.85), while LR provides a competitive, low-complexity alternative for applications that do not require cross-apartment generalization.

Shaoxuan Cui, Lingfei Wang, Hildeberto Jardon-Kojakhmetov et al.

Many complex systems exhibit interactions that depend not only on pairwise connections, but also group structures and memory effects. To capture such effects, we develop a unified tensor framework for modeling higher-order Markov chains with memory. Our formulation introduces an even-order paired tensor that links folded and unfolded dynamics and characterizes their steady states and convergence. We further show that a Markov chain with memory can be approximated by a low-dimensional nonlinear tensor-based system and then provide a full system analysis. As an application, we define random walks on hypergraphs where memory naturally arises from the hyperedge structure, providing new tools for analyzing higher-order networks with time-dependent effects.