Karl H. Johansson

Mahmoud Selim, Amr Alanwar, Shreyas Kousik et al. · gatech

Reinforcement learning (RL) is capable of sophisticated motion planning and control for robots in uncertain environments. However, state-of-the-art deep RL approaches typically lack safety guarantees, especially when the robot and environment models are unknown. To justify widespread deployment, robots must respect safety constraints without sacrificing performance. Thus, we propose a Black-box Reachability-based Safety Layer (BRSL) with three main components: (1) data-driven reachability analysis for a black-box robot model, (2) a trajectory rollout planner that predicts future actions and observations using an ensemble of neural networks trained online, and (3) a differentiable polytope collision check between the reachable set and obstacles that enables correcting unsafe actions. In simulation, BRSL outperforms other state-of-the-art safe RL methods on a Turtlebot 3, a quadrotor, a trajectory-tracking point mass, and a hexarotor in wind with an unsafe set adjacent to the area of highest reward.

Bart Besselink, Karl H. Johansson

A novel delay-based spacing policy for the control of vehicle platoons is introduced together with a notion of disturbance string stability. The delay-based spacing policy specifies the desired inter-vehicular distance between vehicles and guarantees that all vehicles track the same spatially varying reference velocity profile, as is for example required for heavy-duty vehicles driving over hilly terrain. Disturbance string stability is a notion of string stability of vehicle platoons subject to external disturbances on all vehicles that guarantees that perturbations do not grow unbounded as they propagate through the platoon. Specifically, a control design approach in the spatial domain is presented that achieves tracking of the desired spacing policy and guarantees disturbance string stability with respect to a spatially varying reference velocity. The results are illustrated by means of simulations.

Martin Andreasson, Dimos V. Dimarogonas, Henrik Sandberg et al.

This paper analyzes distributed control protocols for first- and second-order networked dynamical systems. We propose a class of nonlinear consensus controllers where the input of each agent can be written as a product of a nonlinear gain, and a sum of nonlinear interaction functions. By using integral Lyapunov functions, we prove the stability of the proposed control protocols, and explicitly characterize the equilibrium set. We also propose a distributed proportional-integral (PI) controller for networked dynamical systems. The PI controllers successfully attenuate constant disturbances in the network. We prove that agents with single-integrator dynamics are stable for any integral gain, and give an explicit tight upper bound on the integral gain for when the system is stable for agents with double-integrator dynamics. Throughout the paper we highlight some possible applications of the proposed controllers by realistic simulations of autonomous satellites, power systems and building temperature control.

Sebastian van de Hoef, Karl H. Johansson, Dimos V. Dimarogonas

The problem of how to coordinate a large fleet of trucks with given itinerary to enable fuel-efficient platooning is considered. Platooning is a promising technology that enables trucks to save significant amounts of fuel by driving close together and thus reducing air drag. A setting is considered in which each truck in a fleet is provided with a start location, a destination, a departure time, and an arrival deadline from a higher planning level. Fuel-efficient plans should be computed. The plans consist of routes and speed profiles that allow trucks to arrive by their arrival deadlines. Hereby, trucks can meet on common parts of their routes and form platoons, resulting in decreased fuel consumption. We formulate a combinatorial optimization problem that combines plans involving only two vehicles. We show that this problem is hard to solve for large problem instances. Hence a heuristic algorithm is proposed. The resulting plans are further optimized using convex optimization techniques. The method is evaluated with Monte Carlo simulations in a realistic setting. We demonstrate that the proposed algorithm can compute plans for thousands of trucks and that significant fuel savings can be achieved.

Farhad Farokhi, Iman Shames, Karl H. Johansson

A conventional way to handle model predictive control (MPC) problems distributedly is to solve them via dual decomposition and gradient ascent. However, at each time-step, it might not be feasible to wait for the dual algorithm to converge. As a result, the algorithm might be needed to be terminated prematurely. One is then interested to see if the solution at the point of termination is close to the optimal solution and when one should terminate the algorithm if a certain distance to optimality is to be guaranteed. In this chapter, we look at this problem for distributed systems under general dynamical and performance couplings, then, we make a statement on validity of similar results where the problem is solved using alternating direction method of multipliers.

Chithrupa Ramesh, Henrik Sandberg, Karl H. Johansson

For a closed-loop system, which has a contention-based multiple access network on its sensor link, the Medium Access Controller (MAC) may discard some packets when the traffic on the link is high. We use a local state-based scheduler to select a few critical data packets to send to the MAC. In this paper, we analyze the impact of such a scheduler on the closed-loop system in the presence of traffic, and show that there is a dual effect with state-based scheduling. In general, this makes the optimal scheduler and controller hard to find. However, by removing past controls from the scheduling criterion, we find that certainty equivalence holds. This condition is related to the classical result of Bar-Shalom and Tse, and it leads to the design of a scheduler with a certainty equivalent controller. This design, however, does not result in an equivalent system to the original problem, in the sense of Witsenhausen. Computing the estimate is difficult, but can be simplified by introducing a symmetry constraint on the scheduler. Based on these findings, we propose a dual predictor architecture for the closed-loop system, which ensures separation between scheduler, observer and controller. We present an example of this architecture, which illustrates a network-aware event-triggering mechanism.

Julien M. Hendrickx, Guodong Shi, Karl H. Johansson

We discuss the possibility of reaching consensus in finite time using only linear iterations, with the additional restrictions that the update matrices must be stochastic with positive diagonals and consistent with a given graph structure. We show that finite-time average consensus can always be achieved for connected undirected graphs. For directed graphs, we show some necessary conditions for finite-time consensus, including strong connectivity and the presence of a simple cycle of even length.

Xiaoqiang Ren, Junfeng Wu, Karl H. Johansson et al.

Jointly optimal transmission power control and remote estimation over an infinite horizon is studied. A sensor observes a dynamic process and sends its observations to a remote estimator over a wireless fading channel characterized by a time-homogeneous Markov chain. The successful transmission probability depends on both the channel gains and the transmission power used by the sensor. The transmission power control rule and the remote estimator should be jointly designed, aiming to minimize an infinite-horizon cost consisting of the power usage and the remote estimation error. A first question one may ask is: Does this joint optimization problem have a solution? We formulate the joint optimization problem as an average cost belief-state Markov decision process and answer the question by proving that there exists an optimal deterministic and stationary policy. We then show that when the monitored dynamic process is scalar, the optimal remote estimates depend only on the most recently received sensor observation, and the optimal transmission power is symmetric and monotonically increasing with respect to the innovation error.

Martina Alutto, Lorenzo Zino, Karl H. Johansson et al.

Leadership in social groups emerges dynamically through interaction and opinion exchange. Empirical evidence indicates that individuals expressing strong opinions tend to gain influence, while sustained leadership critically depends on maintaining alignment with the surrounding social context. Motivated by these observations, we introduce a coupled dynamical model describing the simultaneous evolution of opinions and leadership in a networked population. Extending the Friedkin-Johnsen framework, we represent leadership as a time-varying susceptibility to social influence, which evolves according to a game-theoretic mechanism, consistent with social psychology evidence. Within this setting, agents strengthen their leadership by expressing decisive yet socially coherent opinions, whereas misalignment with the collective state results in a loss of influence. We analyze the coupled dynamics and establish sufficient conditions to identify which agents necessarily emerge as leaders and which act as followers in the social network.

Xinlei Yi, Jieqiang Wei, Dimos V. Dimarogonas et al.

In this paper, event-triggered controllers and corresponding algorithms are proposed to establish the formation with connectivity preservation for multi-agent systems. Each agent needs to update its control input and to broadcast this control input together with the relative state information to its neighbors at its own triggering times, and to receive information at its neighbors' triggering times. Two types of system dynamics, single integrators and double integrators, are considered. As a result, all agents converge to the formation exponentially with connectivity preservation, and Zeno behavior can be excluded. Numerical simulations show the effectiveness of the theoretical results.

Yulong Gao, Karl H. Johansson, Lihua Xie

This paper investigates stochastic invariance for control systems through probabilistic controlled invariant sets (PCISs). As a natural complement to robust controlled invariant sets~(RCISs), we propose finite- and infinite-horizon PCISs, and explore their relation to RICSs. We design iterative algorithms to compute the PCIS within a given set. For systems with discrete spaces, the computations of the finite- and infinite-horizon PCISs at each iteration are based on linear programming and mixed integer linear programming, respectively. The algorithms are computationally tractable and terminate in a finite number of steps. For systems with continuous spaces, we show how to discretize the spaces and prove the convergence of the approximation when computing the finite-horizon PCISs. In addition, it is shown that an infinite-horizon PCIS can be computed by the stochastic backward reachable set from the RCIS contained in it. These PCIS algorithms are applicable to practical control systems. Simulations are given to illustrate the effectiveness of the theoretical results for motion planning.

Farhad Farokhi, Cedric Langbort, Karl H. Johansson

We introduce the family of limited model information control design methods, which construct controllers by accessing the plant's model in a constrained way, according to a given design graph. We investigate the closed-loop performance achievable by such control design methods for fully-actuated discrete-time linear time-invariant systems, under a separable quadratic cost. We restrict our study to control design methods which produce structured static state feedback controllers, where each subcontroller can at least access the state measurements of those subsystems that affect its corresponding subsystem. We compute the optimal control design strategy (in terms of the competitive ratio and domination metrics) when the control designer has access to the local model information and the global interconnection structure of the plant-to-be-controlled. Lastly, we study the trade-off between the amount of model information exploited by a control design method and the best closed-loop performance (in terms of the competitive ratio) of controllers it can produce.

Xiaoqiang Ren, Yilin Mo, Jie Chen et al.

This paper studies static state estimation in multi-sensor settings, with a caveat that an unknown subset of the sensors are compromised by an adversary, whose measurements can be manipulated arbitrarily. The attacker is able to compromise $q$ out of $m$ sensors. A new performance metric, which quantifies the asymptotic decay rate for the probability of having an estimation error larger than $δ$, is proposed. We develop an optimal estimator for the new performance metric with a fixed $δ$, which is the Chebyshev center of a union of ellipsoids. We further provide an estimator that is optimal for every $δ$, for the special case where the sensors are homogeneous. Numerical examples are given to elaborate the results.

Sebastian van de Hoef, Karl H. Johansson, Dimos V. Dimarogonas

The problem of maximizing the probability of two trucks being coordinated to merge into a platoon on a highway is considered. Truck platooning is a promising technology that allows heavy vehicles to save fuel by driving with small automatically controlled inter-vehicle distances. In order to leverage the full potential of platooning, platoons can be formed dynamically en route by small adjustments to their speeds. However, in heavily used parts of the road network, travel times are subject to random disturbances originating from traffic, weather and other sources. We formulate this problem as a stochastic dynamic programming problem over a finite horizon, for which solutions can be computed using a backwards recursion. By exploiting the characteristics of the problem, we derive bounds on the set of states that have to be explored at every stage, which in turn reduces the complexity of computing the solution. Simulations suggest that the approach is applicable to realistic problem instances.

Farhad Farokhi, Karl H. Johansson

Optimal sensor scheduling with applications to networked estimation and control systems is considered. We model sensor measurement and transmission instances using jumps between states of a continuous-time Markov chain. We introduce a cost function for this Markov chain as the summation of terms depending on the average sampling frequencies of the subsystems and the effort needed for changing the parameters of the underlying Markov chain. By minimizing this cost function through extending Brockett's recent approach to optimal control of Markov chains, we extract an optimal scheduling policy to fairly allocate the network resources among the control loops. We study the statistical properties of this scheduling policy in order to compute upper bounds for the closed-loop performance of the networked system, where several decoupled scalar subsystems are connected to their corresponding estimator or controller through a shared communication medium. We generalize the estimation results to observable subsystems of arbitrary order. Finally, we illustrate the developed results numerically on a networked system composed of several decoupled water tanks.

Sebastian van de Hoef, Karl H. Johansson, Dimos V. Dimarogonas

Vehicle platooning facilitates the partial automation of vehicles and can significantly reduce fuel consumption. Mobile communication infrastructure makes it possible to dynamically coordinate the formation of platoons en route. We consider a centralized system that provides trucks with routes and speed profiles allowing them to dynamically form platoons during their journeys. For this to work, all possible pairs of vehicles that can platoon based on their location, destination, and other constraints have to be identified. The presented approach scales well to large vehicle fleets and realistic road networks by extracting features from the transport assignments of the vehicles and rules out a majority of possible pairs based on these features only. Merely a small number of remaining pairs are considered in depth by a complete and computationally expensive algorithm. This algorithm conclusively decides if platooning is possible for a pair based on the complete data associated with the two vehicles. We derive appropriate features for the problem and demonstrate the effectiveness of the approach in a simulation example.

Bart Besselink, Karl H. Johansson, Arjan van der Schaft

This paper introduces assume/guarantee contracts on continuous-time control systems, hereby extending contract theories for discrete systems to certain new model classes and specifications. Contracts are regarded as formal characterizations of control specifications, providing an alternative to specifications in terms of dissipativity properties or set-invariance. The framework has the potential to capture a richer class of specifications more suitable for complex engineering systems. The proposed contracts are supported by results that enable the verification of contract implementation and the comparison of contracts. These results are illustrated by an example of a vehicle following system.

Riccardo Sven Risuleo, Marco Molinari, Giulio Bottegal et al.

This paper describes a benchmark consisting of a set of synthetic measurements relative to an office environment simulated with the software IDA-ICE. The simulated environment reproduces a laboratory at the KTH-EES Smart Building, equipped with a building management system. The data set contains records collected over a period of several days. The signals to CO$_2$ concentration, mechanical ventilation airflows, air infiltrations and occupancy. Information on door and window opening is also available. This benchmark is intended for testing data-based modeling techniques. The ultimate goal is the development of models to improve the forecast and control of environmental variables. Among the numerous challenges related to this framework, we point out the problem of occupancy estimation using information on CO$_2$ concentration. This can be seen as a blind identification problem. For benchmarking purposes, we present two different identification approaches: a baseline overparametrization method and a kernel-based method.

Farhad Farokhi, Karl H. Johansson

The value of plant model information available in the control design process is discussed. We design optimal state-feedback controllers for interconnected discrete-time linear systems with stochastically-varying parameters. The parameters are assumed to be independently and identically distributed random variables in time. The design of each controller relies only on (i) exact local plant model information and (ii) statistical beliefs about the model of the rest of the system. We consider both finite-horizon and infinite-horizon quadratic cost functions. The optimal state-feedback controller is derived in both cases. The optimal controller is shown to be linear in the state and to depend on the model parameters and their statistics in a particular way. Furthermore, we study the value of model information in optimal control design using the performance degradation ratio which is defined as the supremum (over all possible initial conditions) of the ratio of the cost of the optimal controller with limited model information scaled by the cost of the optimal controller with full model information. An upper bound for the performance degradation ratio is presented for the case of fully-actuated subsystems. Comparisons are made between designs based on limited, statistical, and full model information. Throughout the paper, we use a power network example to illustrate concepts and results.

Xinlei Yi, Jieqiang Wei, Karl H. Johansson

The consensus problem for multi-agent systems with quantized communication or sensing is considered. Centralized and distributed self-triggered rules are proposed to reduce the overall need of communication and system updates. It is proved that these self-triggered rules realize consensus exponentially if the network topologies have a spanning tree and the quantization function is uniform. Numerical simulations are provided to show the effectiveness of the theoretical results.

Matin Jafarian, Henrik Sandberg, Karl H. Johansson

This paper studies the stability of voltage dynamics for a power network in which nodal voltages are controlled by means of quadratic droop controllers with nonlinear AC reactive power as inputs. We show that the voltage dynamics is a Lotka-Volterra system, which is a class of nonlinear positive systems. We study the stability of the closed-loop system by proving a uniform ultimate boundedness result and investigating conditions under which the network is cooperative. We then restrict to study the stability of voltage dynamics under a decoupling assumption (i.e., zero relative angles). We analyze the existence and uniqueness of the equilibrium in the interior of the positive orthant for the system and prove an asymptotic stability result.

Mahmoud Selim, Amr Alanwar, M. Watheq El-Kharashi et al.

Reinforcement learning (RL) algorithms can achieve state-of-the-art performance in decision-making and continuous control tasks. However, applying RL algorithms on safety-critical systems still needs to be well justified due to the exploration nature of many RL algorithms, especially when the model of the robot and the environment are unknown. To address this challenge, we propose a data-driven safety layer that acts as a filter for unsafe actions. The safety layer uses a data-driven predictive controller to enforce safety guarantees for RL policies during training and after deployment. The RL agent proposes an action that is verified by computing the data-driven reachability analysis. If there is an intersection between the reachable set of the robot using the proposed action, we call the data-driven predictive controller to find the closest safe action to the proposed unsafe action. The safety layer penalizes the RL agent if the proposed action is unsafe and replaces it with the closest safe one. In the simulation, we show that our method outperforms state-of-the-art safe RL methods on the robotics navigation problem for a Turtlebot 3 in Gazebo and a quadrotor in Unreal Engine 4 (UE4).

Zifan Wang, Yi Shen, Zachary I. Bell et al.

We consider risk-averse learning in repeated unknown games where the goal of the agents is to minimize their individual risk of incurring significantly high cost. Specifically, the agents use the conditional value at risk (CVaR) as a risk measure and rely on bandit feedback in the form of the cost values of the selected actions at every episode to estimate their CVaR values and update their actions. A major challenge in using bandit feedback to estimate CVaR is that the agents can only access their own cost values, which, however, depend on the actions of all agents. To address this challenge, we propose a new risk-averse learning algorithm with momentum that utilizes the full historical information on the cost values. We show that this algorithm achieves sub-linear regret and matches the best known algorithms in the literature. We provide numerical experiments for a Cournot game that show that our method outperforms existing methods.

Phoebus Chen, Chithrupa Ramesh, Karl H. Johansson

Much of the current theory of networked control systems uses simple point-to-point communication models as an abstraction of the underlying network. As a result, the controller has very limited information on the network conditions and performs suboptimally. This work models the underlying wireless multihop mesh network as a graph of links with transmission success probabilities, and uses a recursive Bayesian estimator to provide packet delivery predictions to the controller. The predictions are a joint probability distribution on future packet delivery sequences, and thus capture correlations between successive packet deliveries. We look at finite horizon LQG control over a lossy actuation channel and a perfect sensing channel, both without delay, to study how the controller can compensate for predicted network outages.

Nicola Bastianello, Apostolos I. Rikos, Karl H. Johansson

In this paper we consider online distributed learning problems. Online distributed learning refers to the process of training learning models on distributed data sources. In our setting a set of agents need to cooperatively train a learning model from streaming data. Differently from federated learning, the proposed approach does not rely on a central server but only on peer-to-peer communications among the agents. This approach is often used in scenarios where data cannot be moved to a centralized location due to privacy, security, or cost reasons. In order to overcome the absence of a central server, we propose a distributed algorithm that relies on a quantized, finite-time coordination protocol to aggregate the locally trained models. Furthermore, our algorithm allows for the use of stochastic gradients during local training. Stochastic gradients are computed using a randomly sampled subset of the local training data, which makes the proposed algorithm more efficient and scalable than traditional gradient descent. In our paper, we analyze the performance of the proposed algorithm in terms of the mean distance from the online solution. Finally, we present numerical results for a logistic regression task.

Zifan Wang, Yulong Gao, Siyi Wang et al.

Distributional reinforcement learning (DRL) enhances the understanding of the effects of the randomness in the environment by letting agents learn the distribution of a random return, rather than its expected value as in standard RL. At the same time, a main challenge in DRL is that policy evaluation in DRL typically relies on the representation of the return distribution, which needs to be carefully designed. In this paper, we address this challenge for a special class of DRL problems that rely on linear quadratic regulator (LQR) for control, advocating for a new distributional approach to LQR, which we call \emph{distributional LQR}. Specifically, we provide a closed-form expression of the distribution of the random return which, remarkably, is applicable to all exogenous disturbances on the dynamics, as long as they are independent and identically distributed (i.i.d.). While the proposed exact return distribution consists of infinitely many random variables, we show that this distribution can be approximated by a finite number of random variables, and the associated approximation error can be analytically bounded under mild assumptions. Using the approximate return distribution, we propose a zeroth-order policy gradient algorithm for risk-averse LQR using the Conditional Value at Risk (CVaR) as a measure of risk. Numerical experiments are provided to illustrate our theoretical results.

Matin Jafarian, Mohammad H. Mamduhi, Karl H. Johansson

This paper presents the notion of stochastic phase-cohesiveness based on the concept of recurrent Markov chains and studies the conditions under which a discrete-time stochastic Kuramoto model is phase-cohesive. It is assumed that the exogenous frequencies of the oscillators are combined with random variables representing uncertainties. A bidirectional tree network is considered such that each oscillator is coupled to its neighbors with a coupling law which depends on its own noisy exogenous frequency. In addition, an undirected tree network is studied. For both cases, a sufficient condition for the common coupling strength and a necessary condition for the sampling-period are derived such that the stochastic phase-cohesiveness is achieved. The analysis is performed within the stochastic systems framework and validated by means of numerical simulations.

Farhad Farokhi, Karl H. Johansson

Networked control strategies based on limited information about the plant model usually results in worse closed-loop performance than optimal centralized control with full plant model information. Recently, this fact has been established by utilizing the concept of competitive ratio, which is defined as the worst case ratio of the cost of a control design with limited model information to the cost of the optimal control design with full model information. We show that an adaptive controller, inspired by a controller proposed by Campi and Kumar, with limited plant model information, asymptotically achieves the closed-loop performance of the optimal centralized controller with full model information for almost any plant. Therefore, there exists, at least, one adaptive control design strategy with limited plant model information that can achieve a competitive ratio equal to one. The plant model considered in the paper belongs to a compact set of stochastic linear time-invariant systems and the closed loop performance measure is the ergodic mean of a quadratic function of the state and control input. We illustrate the applicability of the results numerically on a vehicle platooning problem.

Miguel Aguiar, Amritam Das, Karl H. Johansson

We describe a recurrent neural network (RNN) based architecture to learn the flow function of a causal, time-invariant and continuous-time control system from trajectory data. By restricting the class of control inputs to piecewise constant functions, we show that learning the flow function is equivalent to learning the input-to-state map of a discrete-time dynamical system. This motivates the use of an RNN together with encoder and decoder networks which map the state of the system to the hidden state of the RNN and back. We show that the proposed architecture is able to approximate the flow function by exploiting the system's causality and time-invariance. The output of the learned flow function model can be queried at any time instant. We experimentally validate the proposed method using models of the Van der Pol and FitzHugh Nagumo oscillators. In both cases, the results demonstrate that the architecture is able to closely reproduce the trajectories of these two systems. For the Van der Pol oscillator, we further show that the trained model generalises to the system's response with a prolonged prediction time horizon as well as control inputs outside the training distribution. For the FitzHugh-Nagumo oscillator, we show that the model accurately captures the input-dependent phenomena of excitability.

Farhad Farokhi, Henrik Sandberg, Karl H. Johansson

We present a complexity reduction algorithm for a family of parameter-dependent linear systems when the system parameters belong to a compact semi-algebraic set. This algorithm potentially describes the underlying dynamical system with fewer parameters or state variables. To do so, it minimizes the distance (i.e., H-infinity-norm of the difference) between the original system and its reduced version. We present a sub-optimal solution to this problem using sum-of-squares optimization methods. We present the results for both continuous-time and discrete-time systems. Lastly, we illustrate the applicability of our proposed algorithm on numerical examples.

Yu Xing, Xudong Sun, Karl H. Johansson

We study joint learning of network topology and a mixed opinion dynamics, in which agents may have different update rules. Such a model captures the diversity of real individual interactions. We propose a learning algorithm based on multi-armed bandit algorithms to address the problem. The goal of the algorithm is to find each agent's update rule from several candidate rules and to learn the underlying network. At each iteration, the algorithm assumes that each agent has one of the updated rules and then modifies network estimates to reduce validation error. Numerical experiments show that the proposed algorithm improves initial estimates of the network and update rules, decreases prediction error, and performs better than other methods such as sparse linear regression and Gaussian process regression.

Vandana Narri, Jonah J. Glunt, Joshua A. Robbins et al.

Situational awareness for connected and automated vehicles describes the ability to perceive and predict the behavior of other road-users in the near surroundings. However, pedestrians can become occluded by vehicles or infrastructure, creating significant safety risks due to limited visibility. Vehicle-to-everything communication enables the sharing of perception data between connected road-users, allowing for a more comprehensive awareness. The main challenge is how to fuse perception data when measurements are inconsistent with the true locations of pedestrians. Inconsistent measurements can occur due to sensor noise, false positives, or unmodeled disturbances. This paper employs set-based estimation with constrained zonotopes to compute a confidence metric for the measurement set from each sensor. Estimated sets and their confidences are then fused using hybrid zonotopes. This method can account for inconsistent measurements, enabling reliable and robust fusion of the sensor data. The effectiveness of the proposed method is demonstrated in both simulation and real experiments.

Xiaoxing Ren, Yuwen Ma, Nicola Bastianello et al.

We address nonconvex learning problems over undirected networks. In particular, we focus on the challenge of designing an algorithm that is both communication-efficient and that guarantees the privacy of the agents' data. The first goal is achieved through a local training approach, which reduces communication frequency. The second goal is achieved by perturbing gradients during local training, specifically through gradient clipping and additive noise. We prove that the resulting algorithm converges to a stationary point of the problem within a bounded distance. Additionally, we provide theoretical privacy guarantees within a differential privacy framework that ensure agents' training data cannot be inferred from the trained model shared over the network. We show the algorithm's superior performance on a classification task under the same privacy budget, compared with state-of-the-art methods.

Zifan Wang, Riccardo De Santi, Xiaoyu Mo et al.

Fine-tuning pre-trained diffusion and flow models to optimize downstream utilities is central to real-world deployment. Existing entropy-regularized methods primarily maximize expected reward, providing no mechanism to shape tail behavior. However, tail control is often essential: the lower tail determines reliability by limiting low-reward failures, while the upper tail enables discovery by prioritizing rare, high-reward outcomes. In this work, we present Tail-aware Flow Fine-Tuning (TFFT), a principled and efficient distributional fine-tuning algorithm based on the Conditional Value-at-Risk (CVaR). We address two distinct tail-shaping goals: right-CVaR for seeking novel samples in the high-reward tail and left-CVaR for controlling worst-case samples in the low-reward tail. Unlike prior approaches that rely on non-linear optimization, we leverage the variational dual formulation of CVaR to decompose it into a decoupled two-stage procedure: a lightweight one-dimensional threshold optimization step, and a single entropy-regularized fine-tuning process via a specific pseudo-reward. This decomposition achieves CVaR fine-tuning efficiently with computational cost comparable to standard expected fine-tuning methods. We demonstrate the effectiveness of TFFT across illustrative experiments, high-dimensional text-to-image generation, and molecular design.

Xu Du, Boyu Han, Ivano Notarnicola et al.

This paper investigates distributed resource allocation optimization over directed graphs with limited communication bandwidth. We develop a novel distributed algorithm that integrates the centralized Proximal Jacobian Alternating Direction Method of Multipliers (PJ-ADMM) with a finite-level quantized consensus scheme, enabling nodes to cooperatively solve the optimization in a distributed fashion. Under the assumption of convex objective functions, we establish that the proposed algorithm achieves sublinear convergence to a neighborhood of the optimal solution, with the convergence accuracy explicitly bounded by the quantization level. Numerical experiments validate that the algorithm achieves competitive performance compared to existing approaches while exhibiting communication efficiency.

Behrad Samari, Henrik Sandberg, Karl H. Johansson et al.

Model order reduction techniques simplify high-dimensional dynamical systems by deriving lower-dimensional models that retain essential system characteristics. These techniques are crucial for the controller design of complex systems while significantly reducing computational costs. Nevertheless, constructing effective reduced-order models (ROMs) poses considerable challenges, particularly for nonlinear dynamical systems. These challenges are further exacerbated when the actual system model is unavailable, a scenario frequently encountered in real-world applications. In this work, we propose a data-driven framework for constructing ROMs of nonlinear dynamical systems with unknown mathematical models, enabling controller synthesis directly from the resulting ROMs. We establish similarity relations between the output trajectories of the original systems and those of their ROMs by employing the notion of simulation functions (SFs), thereby enabling a formal characterization of their closeness. To achieve this, we collect one set of noise-corrupted input-state data from the system during a finite-time experiment, upon which we propose conditions to construct both ROMs and SFs simultaneously. These conditions are formulated as data-dependent semidefinite programs. We demonstrate that the data-driven ROMs obtained can be employed to synthesize controllers for the original unknown systems, ensuring that they satisfy high-level logic specifications. This is accomplished by first designing controllers for the data-driven ROMs and then translating the results back to the original systems via interface functions, designed directly from the proposed data-dependent conditions. We evaluate the efficacy of our data-driven framework through two case studies, including a challenging benchmark from the model reduction literature: a circuit of chained inverter gates with 20 state variables.

Behrad Samari, Henrik Sandberg, Karl H. Johansson et al.

This paper develops a direct data-driven framework for constructing reduced-order models (ROMs) of discrete-time linear dynamical systems with unknown dynamics and process disturbances. The proposed scheme enables controller synthesis on the ROM and its refinement to the original system by an interface function designed using noisy data. To achieve this, the notion of simulation functions (SFs) is employed to establish a formal relation between the original system and its ROM, yielding a quantitative bound on the mismatch between their output trajectories. To construct such relations and interface functions, we rely on data collected from the unknown system. In particular, using noise-corrupted input-state data gathered along a single trajectory of the system, and without identifying the original dynamics, we propose data-dependent conditions, cast as a semidefinite program, for the simultaneous construction of ROMs, SFs, and interface functions. Through a case study, we demonstrate that data-driven controller synthesis on the ROM, combined with controller refinement via the interface function, enables the enforcement of complex specifications beyond stability.

Seyed Mohammad Azimi-Abarghouyi, Nicola Bastianello, Karl H. Johansson et al.

In this paper, we depart from the widely-used gradient descent-based hierarchical federated learning (FL) algorithms to develop a novel hierarchical FL framework based on the alternating direction method of multipliers (ADMM). Within this framework, we propose two novel FL algorithms, which both use ADMM in the top layer: one that employs ADMM in the lower layer and another that uses the conventional gradient descent-based approach. The proposed framework enhances privacy, and experiments demonstrate the superiority of the proposed algorithms compared to the conventional algorithms in terms of learning convergence and accuracy. Additionally, gradient descent on the lower layer performs well even if the number of local steps is very limited, while ADMM on both layers lead to better performance otherwise.

Ruochen Wu, Xu Du, Karl H. Johansson et al.

In modern large-scale networked systems, rapidly solving optimization problems while utilizing communication resources efficiently is critical for addressing complex tasks. In this paper, we consider an unconstrained distributed optimization problem in which information exchange among nodes is governed by a directed communication graph. In our setup we focus on two key challenges. The first is the zigzag phenomenon caused by the objective functions of individual nodes having significantly different curvature along different directions. The second is that the communication channels among nodes are subject to limited bandwidth, which motivates the use of compressed (quantized) messages. To address both challenges simultaneously, we propose QANM, a distributed optimization algorithm that combines Nesterov-accelerated gradient descent with a distributed finite-time quantized consensus protocol, enabling accelerated convergence. Under strong convexity and smoothness assumptions, we show that our proposed algorithm converges linearly to a neighborhood of the optimal solution. Finally, we validate our algorithm on a distributed sensor fusion application for multi-dimensional target parameter estimation, where simulations across two distinct scenarios confirm the convergence guarantees and demonstrate clear acceleration benefits over non-momentum baselines.

Boyu Han, Xu Du, Karl H. Johansson et al.

This paper addresses distributed consensus optimization problems with mixed-integer variables, with a specific focus on Boolean variables. We introduce a novel distributed algorithm that extends the Consensus Augmented Lagrangian Alternating Direction Inexact Newton (CALADIN) framework by incorporating specialized techniques for handling Boolean variables without relying on local mixed-integer solvers. Under the mild assumption of Lipschitz continuity of the objective functions, we establish rigorous convergence guarantees for both convex and nonconvex mixed-integer programming problems. Numerical experiments demonstrate that the proposed algorithm achieves competitive performance compared to existing approaches while providing rigorous convergence guarantees.

Elis Stefansson, Karl H. Johansson

We consider optimal planning in a large-scale system formalised as a hierarchical finite state machine (HFSM). A planning algorithm is proposed computing an optimal plan between any two states in the HFSM, consisting of two steps: A pre-processing step that computes optimal exit costs of the machines in the HFSM, with time complexity scaling with the number of machines; and a query step that efficiently computes an optimal plan by removing irrelevant subtrees of the HFSM using the optimal exit costs. The algorithm is reconfigurable in the sense that changes in the HFSM are handled with ease, where the pre-processing step recomputes only the optimal exit costs affected by the change. The algorithm can also exploit compact representations that groups together identical machines in the HFSM, where the algorithm only needs to compute the optimal exit costs for one of the identical machines within each group, thereby avoid unnecessary recomputations. We validate the algorithm on large systems with millions of states and a robotic application. It is shown that our approach outperforms Dijkstra's algorithm, Bidirectional Dijkstra and Contraction Hierarchies.

Xu Du, Jingzhe Wang, Karl H. Johansson et al.

Distributed optimization has found widespread applications in smart grids, optimal control, and machine learning. This paper studies distributed consensus optimization. We extend the Augmented Lagrangian-based Alternating Direction Inexact Newton (ALADIN) framework to propose Consensus ALADIN (C-ALADIN) with a central coordinator, which directly handles consensus constraints. Our C-ALADIN algorithm admits both a first-order variant and a second-order variant that employs a Hessian approximation, avoiding direct transmission of second-order information while preserving fast local convergence. We then develop a decentralized version of C-ALADIN that operates over directed graphs with quantized communication, using a finite-time coordination protocol. For both versions, we establish global convergence guarantees for convex problems and local convergence guarantees for non-convex problems. For the decentralized case, the iterates converge to a neighborhood of the optimum determined by the quantization level. Numerical results demonstrate that our methods retain fast convergence while substantially reducing communication and computational costs compared to existing decentralized approaches.

Changxin Liu, Nicola Bastianello, Wei Huo et al.

Decentralized optimization has become a standard paradigm for solving large-scale decision-making problems and training large machine learning models without centralizing data. However, this paradigm introduces new privacy and security risks, with malicious agents potentially able to infer private data or impair the model accuracy. Over the past decade, significant advancements have been made in developing secure decentralized optimization and learning frameworks and algorithms. This survey provides a comprehensive tutorial on these advancements. We begin with the fundamentals of decentralized optimization and learning, highlighting centralized aggregation and distributed consensus as key modules exposed to security risks in federated and distributed optimization, respectively. Next, we focus on privacy-preserving algorithms, detailing three cryptographic tools and their integration into decentralized optimization and learning systems. Additionally, we examine resilient algorithms, exploring the design and analysis of resilient aggregation and consensus protocols that support these systems. We conclude the survey by discussing current trends and potential future directions.

Zhen Zhang, M. Umar B. Niazi, Michelle S. Chong et al.

This paper studies deterministic data-driven reachability analysis for dynamical systems with unknown dynamics and nonconvex reachable sets. Existing deterministic data-driven approaches typically employ zonotopic set representations, for which the multiplication between a zonotopic model set and a zonotopic state set cannot be represented algebraically exactly, thereby necessitating over-approximation steps in reachable-set propagation. To remove this structural source of conservatism, we introduce constrained polynomial matrix zonotopes (CPMZs) to represent data-consistent model sets, and show that the multiplication between a CPMZ model set and a constrained polynomial zonotope (CPZ) state set admits an algebraically exact CPZ representation. This property enables set propagation entirely within the CPZ representation, thereby avoiding propagation-induced over-approximation and even retaining the ability to represent nonconvex reachable sets. Moreover, we develop set-theoretic results that enable the intersection of data-consistent model sets as new data become available, yielding the proposed online refinement scheme that progressively tightens the data-consistent model set and, in turn, the resulting reachable set. Beyond linear systems, we extend the proposed framework to polynomial dynamics and develop additional set-theoretic results that enable both model-based and data-driven reachability analysis within the same algebraic representation. By deriving algebraically exact CPZ representations for monomials and their compositions, reachable-set propagation can be carried out directly at the set level without resorting to interval arithmetic or relaxation-based bounding techniques. Numerical examples for both linear and polynomial systems demonstrate a significant reduction in conservatism compared to state-of-the-art deterministic data-driven reachability methods.

Marijn Ruiter, Miguel Aguiar, Jake Rap et al.

We propose RHYME-XT, an operator-learning framework for surrogate modeling of spatiotemporal control systems governed by input-affine nonlinear partial integro-differential equations (PIDEs) with localized rhythmic behavior. RHYME-XT uses a Galerkin projection to approximate the infinite-dimensional PIDE on a learned finite-dimensional subspace with spatial basis functions parameterized by a neural network. This yields a projected system of ODEs driven by projected inputs. Instead of integrating this non-autonomous system, we directly learn its flow map using an architecture for learning flow functions, avoiding costly computations while obtaining a continuous-time and discretization-invariant representation. Experiments on a neural field PIDE show that RHYME-XT outperforms a state-of-the-art neural operator and is able to transfer knowledge effectively across models trained on different datasets, through a fine-tuning process.

Anqi Dong, Yongxin Chen, Karl H. Johansson et al.

Steering large-scale swarms in only a few control updates is challenging because real systems operate in sampled-data form: control inputs are updated intermittently and applied over finite intervals. In this regime, the natural object is not an instantaneous velocity field, but a finite-window control quantity that captures the system response over each sampling interval. Inspired by MeanFlow, we introduce a control-space learning framework for swarm steering under linear time-invariant dynamics. The learned object is the coefficient that parameterizes the finite-horizon minimum-energy control over each interval. We show that this coefficient admits both an integral representation and a local differential identity along bridge trajectories, which leads to a simple stop-gradient training objective. At implementation time, the learned coefficient is used directly in sampled-data updates, so the prescribed dynamics and actuation map are respected by construction. The resulting framework provides a scalable approach to few-step swarm steering that is consistent with the sampled-data structure of real control systems.

Alejandro Penacho Riveiros, Nicola Bastianello, Karl H. Johansson et al.

As the number of satellites in orbit has increased exponentially in recent years, ensuring their correct functionality has started to require automated methods to decrease human workload. In this work, we present an algorithm that analyzes the on-board data related to friction from the Reaction Wheel Assemblies (RWA) of a satellite and determines their operating status, distinguishing between nominal status and several possible anomalies that require preventive measures to be taken. The algorithm first uses a model based on hybrid systems theory to extract the information relevant to the problem. The extraction process combines techniques in changepoint detection, dynamic programming, and maximum likelihood in a structured way. A classifier then uses the extracted information to determine the status of the RWA. This last classifier has been previously trained with a labelled dataset produced by a high-fidelity simulator, comprised for the most part of nominal data. The final algorithm combines model-based and data-based approaches to obtain satisfactory results with an accuracy around 95%.

Zishuo Li, Anh Tung Nguyen, André M. H. Teixeira et al.

This paper addresses the secure state estimation problem for continuous linear time-invariant systems with non-periodic and asynchronous sampled measurements, where the sensors need to transmit not only measurements but also sampling time-stamps to the fusion center. This measurement and communication setup is well-suited for operating large-scale control systems and, at the same time, introduces new vulnerabilities that can be exploited by adversaries through (i) manipulation of measurements, (ii) manipulation of time-stamps, (iii) elimination of measurements, (iv) generation of completely new false measurements, or a combination of these attacks. To mitigate these attacks, we propose a decentralized estimation algorithm in which each sensor maintains its local state estimate asynchronously based on its measurements. The local states are synchronized through time prediction and fused after time-stamp alignment. In the absence of attacks, state estimates are proven to recover the optimal Kalman estimates by solving a weighted least square problem. In the presence of attacks, solving this weighted least square problem with the aid of $\ell_1$ regularization provides secure state estimates with uniformly bounded error under an observability redundancy assumption. The effectiveness of the proposed algorithm is demonstrated using a benchmark example of the IEEE 14-bus system.

Yu Xing, Aneesh Raghavan, Michael T. Schaub et al.

We consider a verification problem for opinion dynamics based on binary observations. The opinion dynamics is governed by a Friedkin-Johnsen (FJ) model, where only a sequence of binary outputs is available instead of the agents' continuous opinions. Specifically, at every time-step we observe a binarized output for each agent depending on whether the opinion exceeds a fixed threshold. The objective is to verify whether an FJ model with a given set of stubbornness parameters and initial opinions is consistent with the observed binary outputs up to a small error. The FJ model is formulated as a transition system, and an approximate simulation relation of two transition systems is defined in terms of the proximity of their opinion trajectories and output sequences. We then construct a finite set of abstract FJ models by simplifying the influence matrix and discretizing the stubbornness parameters and the initial opinions. It is shown that the abstraction approximately simulates any concrete FJ model with continuous parameters and initial opinions, and is itself approximately simulated by some concrete FJ model. These results ensure that consistency verification can be performed over the finite abstraction. Specifically, by checking whether an abstract model satisfies the observation constraints, we can conclude whether the corresponding family of concrete FJ models is consistent with the binary observations. Finally, numerical experiments are presented to illustrate the proposed verification framework.

Kleio Fragkedaki, Frank J. Jiang, Karl H. Johansson et al.

In this work, we present a transformer-based framework for predicting future pedestrian states based on clustered historical trajectory data. In previous studies, researchers propose enhancing pedestrian trajectory predictions by using manually crafted labels to categorize pedestrian behaviors and intentions. However, these approaches often only capture a limited range of pedestrian behaviors and introduce human bias into the predictions. To alleviate the dependency on manually crafted labels, we utilize a transformer encoder coupled with hierarchical density-based clustering to automatically identify diverse behavior patterns, and use these clusters in data-driven reachability analysis. By using a transformer-based approach, we seek to enhance the representation of pedestrian trajectories and uncover characteristics or features that are subsequently used to group trajectories into different "behavior" clusters. We show that these behavior clusters can be used with data-driven reachability analysis, yielding an end-to-end data-driven approach to predicting the future motion of pedestrians. We train and evaluate our approach on a real pedestrian dataset, showcasing its effectiveness in forecasting pedestrian movements.