Murat Arcak

Majid Zamani, Murat Arcak

In this paper we propose a compositional scheme for the construction of abstractions for networks of control systems using the interconnection matrix and joint dissipativity-type properties of subsystems and their abstractions. In the proposed framework, the abstraction, itself a control system (possibly with a lower dimension), can be used as a substitution of the original system in the controller design process. Moreover, we provide a procedure for constructing abstractions of a class of nonlinear control systems by using the bounds on the slope of system nonlinearities. We illustrate the proposed results on a network of linear control systems by constructing its abstraction in a compositional way without requiring any condition on the number or gains of the subsystems. We use the abstraction as a substitute to synthesize a controller enforcing a certain linear temporal logic specification. This example particularly elucidates the effectiveness of dissipativity-type compositional reasoning for large-scale systems.

Pierre-Jean Meyer, Alex Devonport, Murat Arcak

This paper presents TIRA, a Matlab library gathering several methods for the computation of interval over-approximations of the reachable sets for both continuous- and discrete-time nonlinear systems. Unlike other existing tools, the main strength of interval-based reachability analysis is its simplicity and scalability, rather than the accuracy of the over-approximations. The current implementation of TIRA contains four reachability methods covering wide classes of nonlinear systems, handled with recent results relying on contraction/growth bounds and monotonicity concepts. TIRA's architecture features a central function working as a hub between the user-defined reachability problem and the library of available reachability methods. This design choice offers increased extensibility of the library, where users can define their own method in a separate function and add the function call in the hub function.

Samuel Coogan, Murat Arcak, Alexander A. Kurzhanskiy

In vehicle traffic networks, congestion on one outgoing link of a diverging junction often impedes flow to other outgoing links, a phenomenon known as the first-in-first-out (FIFO) property. Simplified traffic models that do not account for the FIFO property result in monotone dynamics for which powerful analysis techniques exist. FIFO models are in general not monotone, but have been shown to be mixed monotone - a generalization of monotonicity that enables similarly powerful analysis techniques. In this paper, we study traffic flow models for which the FIFO property is only partial, that is, flows at diverging junctions exhibit a combination of FIFO and non-FIFO phenomena. We show that mixed monotonicity extends to this wider class of models and establish conditions that guarantee convergence to an equilibrium.

Ana Sofia Rufino Ferreira, Justin Hsia, Murat Arcak et al.

We propose a compartmental lateral inhibition system that generates contrasting patterns of gene expression between neighboring compartments. The system consists of a set of compartments interconnected by channels. Each compartment contains a colony of cells that produce diffusible molecules to be detected by the neighboring colony, and each cell is equipped with an inhibitory circuit that reduces its production when the detected signal is stronger. We develop a technique to analyze the steady-state patterns emerging from this lateral inhibition system and apply it to a specific implementation. The analysis shows that the proposed system indeed exhibits contrasting patterns within realistic parameter ranges.

Ana S. Rufino Ferreira, Murat Arcak

We analyze pattern formation on a network of cells where each cell inhibits its neighbors through cell-to-cell contact signaling. The network is modeled as an interconnection of identical dynamical subsystems each of which represents the signaling reactions in a cell. We search for steady state patterns by partitioning the graph vertices into disjoint classes, where the cells in the same class have the same final fate. To prove the existence of steady states with this structure, we use results from monotone systems theory. Finally, we analyze the stability of these patterns with a block decomposition based on the graph partition.

Murat Arcak, John Maidens

A shortcoming of existing reachability approaches for nonlinear systems is the poor scalability with the number of continuous state variables. To mitigate this problem we present a simulation-based approach where we first sample a number of trajectories of the system and next establish bounds on the convergence or divergence between the samples and neighboring trajectories. We compute these bounds using contraction theory and reduce the conservatism by partitioning the state vector into several components and analyzing contraction properties separately in each direction. Among other benefits this allows us to analyze the effect of constant but uncertain parameters by treating them as state variables and partitioning them into a separate direction. We next present a numerical procedure to search for weighted norms that yield a prescribed contraction rate, which can be incorporated in the reachability algorithm to adjust the weights to minimize the growth of the reachable set.

John Maidens, Murat Arcak

We present a method of computing backward reachable sets for nonlinear discrete-time control systems possessing continuous symmetries. The starting point is a dynamic game formulation of reachability analysis where control inputs aim to maintain the state variables within a target tube despite disturbances. Our method exploits symmetry to compute the reachable sets in a lower-dimensional space, enabling a significant computational speedup. To achieve this, we present a general method for symmetry reduction based on the Cartan frame, which simplifies the dynamic programming iteration without algebraic manipulation of the state update equations. We illustrate the results by computing a backward reachable set for a six-dimensional reach-avoid game of two Dubins vehicles.

John Maidens, Murat Arcak

We consider the problem of optimal input design for estimating uncertain parameters in a discrete-time linear state space model, subject to simultaneous amplitude and l1/l2-norm constraints on the admissible inputs. We formulate this problem as the maximization of a (non-concave) quadratic function over the space of inputs, and use semidefinite relaxation techniques to efficiently find the global solution or to provide an upper bound. This investigation is motivated by a problem in medical imaging, specifically designing a substrate injection profile for in vivo metabolic parameter mapping using magnetic resonance imaging (MRI) with hyperpolarized carbon-13 pyruvate. In the l2-norm-constrained case, we show that the relaxation is tight, allowing us to efficiently compute a globally optimal injection profile. In the l1-norm-constrained case the relaxation is no longer tight, but can be used to prove that the boxcar injection currently used in practice achieves at least 98.7% of the global optimum.

John Maidens, Axel Barrau, Silvere Bonnabel et al.

We present a method of exploiting symmetries of discrete-time optimal control problems to reduce the dimensionality of dynamic programming iterations. The results are derived for systems with continuous state variables, and can be applied to systems with continuous or discrete symmetry groups. We prove that symmetries of the state update equation and stage costs induce corresponding symmetries of the optimal cost function and the optimal policies. We then provide a general framework for computing the optimal cost function based on gridding a space of lower dimension than the original state space. This method does not require algebraic manipulation of the state update equations; it only requires knowledge of the symmetries that the state update equations possess. Since the method can be performed without any knowledge of the state update map beyond being able to evaluate it and verify its symmetries, this enables the method to be applied in a wide range of application problems. We illustrate these results on two six-dimensional optimal control problems that are computationally difficult to solve by dynamic programming without symmetry reduction.

Samuel Coogan, Murat Arcak

In this note, we propose a case study of freeway traffic flow modeled as a hybrid system. We describe two general classes of networks that model flow along a freeway with merging onramps. The admission rate of traffic flow from each onramp is metered via a control input. Both classes of networks are easily scaled to accommodate arbitrary state dimension. The model is discrete-time and possesses piecewise-affine dynamics. Moreover, we present several control objectives that are especially relevant for traffic flow management. The proposed model is flexible and extensible and offers a benchmark for evaluating tools and techniques developed for hybrid systems.

Emmanuel Sin, Murat Arcak, Andrew Packard

We study the optimal control of an arbitrarily large constellation of small satellites operating in low Earth orbit. Simulating the lack of on-board propulsion, we limit our actuation to the use of differential drag maneuvers to make in-plane changes to the satellite orbits. We propose an efficient method to separate a cluster of satellites into a desired constellation shape while respecting actuation constraints and maximizing the operational lifetime of the constellation. By posing the problem as a linear program, we solve for the optimal drag commands for each of the satellites on a daily basis with a shrinking-horizon model predictive control approach. We then apply this control strategy in a nonlinear orbital dynamics simulation with a simple, varying atmospheric density model. We demonstrate the ability to control a cluster of 100+ satellites starting at the same initial conditions in a circular low Earth orbit to form an equally spaced constellation (with a relative angular separation error tolerance of one-tenth a degree). The constellation separation task can be executed in 71 days, a time frame that is competitive for the state-of-the-practice. This method allows us to trade the time required to converge to the desired constellation with a sacrifice in the overall constellation lifetime, measured as the maximum altitude loss experienced by one of the satellites in the group after the separation maneuvers.

Eric S. Kim, Murat Arcak, Sanjit A. Seshia

Successfully synthesizing controllers for complex dynamical systems and specifications often requires leveraging domain knowledge as well as making difficult computational or mathematical tradeoffs. This paper presents a flexible and extensible framework for constructing robust control synthesis algorithms and applies this to the traditional abstraction-based control synthesis pipeline. It is grounded in the theory of relational interfaces and provides a principled methodology to seamlessly combine different techniques (such as dynamic precision grids, refining abstractions while synthesizing, or decomposed control predecessors) or create custom procedures to exploit an application's intrinsic structural properties. A Dubins vehicle is used as a motivating example to showcase memory and runtime improvements.

Eric S. Kim, Murat Arcak

Translating continuous control system models into finite automata allows us to use powerful discrete tools to synthesize controllers for complex specifications. The abstraction construction step is unfortunately hamstrung by high runtime and memory requirements for high dimensional systems. This paper describes how the transition relation encoding the abstract system dynamics can be generated by connecting smaller abstract modules in series and parallel. We provide a composition operation and show that composing a collection of abstract modules yields another abstraction satisfying a feedback refinement relation. Through compositionality we circumvent the acute computational cost of directly abstracting a high dimensional system and also modularize the abstraction construction pipeline.

Paul Griffioen, Bingzhuo Zhong, Murat Arcak et al.

We compute probabilistic controlled invariant sets for nonlinear systems using Gaussian process state space models, which are data-driven models that account for unmodeled and unknown nonlinear dynamics. We propose a semidefinite programming scheme for designing state-feedback controllers that maximize the probability of the trajectories staying within a probabilistic controlled invariant set while satisfying input constraints. The results are validated on a quadrotor, both in simulation and on a physical platform.

Samuel Coogan, Murat Arcak

In this note, we study Luenberger-type full-state observers for nonlinear systems using contraction theory. We show that if the matrix measure of a suitably defined Jacobian matrix constructed from the dynamics of the system-observer interconnection is uniformly negative, then the state estimate converges exponentially to the actual state. This sufficient condition for convergence establishes that the distance between the estimate and state is infinitesimally contracting with respect to some norm on the state-space. In contrast to existing results for contraction-based observer design, we allow for contraction with respect to non-Euclidean norms. Such norms have proven useful in applications. To demonstrate our results, we study the problem of observing vehicular traffic density along a freeway modeled as interconnected, spatially homogenous compartments, and our approach relies on establishing contraction of the system-observer interconnection with respect to the one-norm.

Semih Kara, Yasin Sonmez, Can Kizilkale et al.

Growing EV adoption can worsen traffic conditions if chargers are sited without regard to their impact on congestion. We study how to strategically place EV chargers to reduce congestion using two equilibrium models: one based on congestion games and one based on an atomic queueing simulation. We apply both models within a scalable greedy station-placement algorithm. Experiments show that this greedy scheme yields optimal or near-optimal congestion outcomes in realistic networks, even though global optimality is not guaranteed as we show with a counterexample. We also show that the queueing-based approach yields more realistic results than the congestion-game model, and we present a unified methodology that calibrates congestion delays from queue simulation and solves equilibrium in link-space.

Neelay Junnarkar, Murat Arcak, Peter Seiler

We present a method to train neural network controllers with guaranteed stability margins. The method is applicable to linear time-invariant plants interconnected with uncertainties and nonlinearities that are described by integral quadratic constraints. The type of stability margin we consider is the disk margin. Our training method alternates between a training step to maximize reward and a stability margin-enforcing step. In the stability margin enforcing-step, we solve a semidefinite program to project the controller into the set of controllers for which we can certify the desired disk margin.

Neelay Junnarkar, Murat Arcak, Peter Seiler

In this paper, a method is presented to synthesize neural network controllers such that the feedback system of plant and controller is dissipative, certifying performance requirements such as L2 gain bounds. The class of plants considered is that of linear time-invariant (LTI) systems interconnected with an uncertainty, including nonlinearities treated as an uncertainty for convenience of analysis. The uncertainty of the plant and the nonlinearities of the neural network are both described using integral quadratic constraints (IQCs). First, a dissipativity condition is derived for uncertain LTI systems. Second, this condition is used to construct a linear matrix inequality (LMI) which can be used to synthesize neural network controllers. Finally, this convex condition is used in a projection-based training method to synthesize neural network controllers with dissipativity guarantees. Numerical examples on an inverted pendulum and a flexible rod on a cart are provided to demonstrate the effectiveness of this approach.

Yasin Sonmez, Neelay Junnarkar, Murat Arcak

Recent work in reinforcement learning has leveraged symmetries in the model to improve sample efficiency in training a policy. A commonly used simplifying assumption is that the dynamics and reward both exhibit the same symmetry; however, in many real-world environments, the dynamical model exhibits symmetry independent of the reward model. In this paper, we assume only the dynamics exhibit symmetry, extending the scope of problems in reinforcement learning and learning in control theory to which symmetry techniques can be applied. We use Cartan's moving frame method to introduce a technique for learning dynamics that, by construction, exhibit specified symmetries. Numerical experiments demonstrate that the proposed method learns a more accurate dynamical model

Marlon Müller, Florian Finkeldei, Hanna Krasowski et al.

Compliance with maritime traffic rules is essential for the safe operation of autonomous vessels, yet training reinforcement learning (RL) agents to adhere to them is challenging. The behavior of RL agents is shaped by the training scenarios they encounter, but creating scenarios that capture the complexity of maritime navigation is non-trivial, and real-world data alone is insufficient. To address this, we propose a falsification-driven RL approach that generates adversarial training scenarios in which the vessel under test violates maritime traffic rules, which are expressed as signal temporal logic specifications. Our experiments on open-sea navigation with two vessels demonstrate that the proposed approach provides more relevant training scenarios and achieves more consistent rule compliance.

Eric Palanques-Tost, Hanna Krasowski, Murat Arcak et al.

Biomolecular Neural Networks (BNNs), artificial neural networks with biologically synthesizable architectures, achieve universal function approximation capabilities beyond simple biological circuits. However, training BNNs remains challenging due to the lack of target data. To address this, we propose leveraging Signal Temporal Logic (STL) specifications to define training objectives for BNNs. We build on the quantitative semantics of STL, enabling gradient-based optimization of the BNN weights, and introduce a learning algorithm that enables BNNs to perform regression and control tasks in biological systems. Specifically, we investigate two regression problems in which we train BNNs to act as reporters of dysregulated states, and a feedback control problem in which we train the BNN in closed-loop with a chronic disease model, learning to reduce inflammation while avoiding adverse responses to external infections. Our numerical experiments demonstrate that STL-based learning can solve the investigated regression and control tasks efficiently.

Baturalp Yalcin, Haixiang Zhang, Javad Lavaei et al.

This paper investigates the system identification problem for linear discrete-time systems under adversaries and analyzes two lasso-type estimators. We examine both asymptotic and non-asymptotic properties of these estimators in two separate scenarios, corresponding to deterministic and stochastic models for the attack times. Since the samples collected from the system are correlated, the existing results on lasso are not applicable. We prove that when the system is stable and attacks are injected periodically, the sample complexity for exact recovery of the system dynamics is linear in terms of the dimension of the states. When adversarial attacks occur at each time instance with probability p, the required sample complexity for exact recovery scales polynomially in the dimension of the states and the probability p. This result implies almost sure convergence to the true system dynamics under the asymptotic regime. As a by-product, our estimators still learn the system correctly even when more than half of the data is compromised. We highlight that the attack vectors are allowed to be correlated with each other in this work, whereas we make some assumptions about the times at which the attacks happen. This paper provides the first mathematical guarantee in the literature on learning from correlated data for dynamical systems in the case when there is less clean data than corrupt data.

Neelay Junnarkar, He Yin, Fangda Gu et al.

We propose a parameterization of a nonlinear dynamic controller based on the recurrent equilibrium network, a generalization of the recurrent neural network. We derive constraints on the parameterization under which the controller guarantees exponential stability of a partially observed dynamical system with sector bounded nonlinearities. Finally, we present a method to synthesize this controller using projected policy gradient methods to maximize a reward function with arbitrary structure. The projection step involves the solution of convex optimization problems. We demonstrate the proposed method with simulated examples of controlling nonlinear plants, including plants modeled with neural networks.

Alex Devonport, Forest Yang, Laurent El Ghaoui et al.

We present algorithms for estimating the forward reachable set of a dynamical system using only a finite collection of independent and identically distributed samples. The produced estimate is the sublevel set of a function called an empirical inverse Christoffel function: empirical inverse Christoffel functions are known to provide good approximations to the support of probability distributions. In addition to reachability analysis, the same approach can be applied to general problems of estimating the support of a random variable, which has applications in data science towards detection of novelties and outliers in data sets. In applications where safety is a concern, having a guarantee of accuracy that holds on finite data sets is critical. In this paper, we prove such bounds for our algorithms under the Probably Approximately Correct (PAC) framework. In addition to applying classical Vapnik-Chervonenkis (VC) dimension bound arguments, we apply the PAC-Bayes theorem by leveraging a formal connection between kernelized empirical inverse Christoffel functions and Gaussian process regression models. The bound based on PAC-Bayes applies to a more general class of Christoffel functions than the VC dimension argument, and achieves greater sample efficiency in experiments.

Fangda Gu, He Yin, Laurent El Ghaoui et al.

Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stability during the learning and control process, we propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space taking advantage of mild additional information on system dynamics. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.

Alex Devonport, Adnane Saoud, Murat Arcak

Symbolic control techniques aim to satisfy complex logic specifications. A critical step in these techniques is the construction of a symbolic (discrete) abstraction, a finite-state system whose behaviour mimics that of a given continuous-state system. The methods used to compute symbolic abstractions, however, require knowledge of an accurate closed-form model. To generalize them to systems with unknown dynamics, we present a new data-driven approach that does not require closed-form dynamics, instead relying only the ability to evaluate successors of each state under given inputs. To provide guarantees for the learned abstraction, we use the Probably Approximately Correct (PAC) statistical framework. We first introduce a PAC-style behavioural relationship and an appropriate refinement procedure. We then show how the symbolic abstraction can be constructed to satisfy this new behavioural relationship. Moreover, we provide PAC bounds that dictate the number of data required to guarantee a prescribed level of accuracy and confidence. Finally, we present an illustrative example.

Yash Vardhan Pant, He Yin, Murat Arcak et al.

Urban Air Mobility (UAM), or the scenario where multiple manned and Unmanned Aerial Vehicles (UAVs) carry out various tasks over urban airspaces, is a transportation concept of the future that is gaining prominence. UAM missions with complex spatial, temporal and reactive requirements can be succinctly represented using Signal Temporal Logic (STL), a behavioral specification language. However, planning and control of systems with STL specifications is computationally intensive, usually resulting in planning approaches that do not guarantee dynamical feasibility, or control approaches that cannot handle complex STL specifications. Here, we present an approach to co-design the planner and control such that a given STL specification (possibly over multiple UAVs) is satisfied with trajectories that are dynamically feasible and our controller can track them with a bounded tracking-error that the planner accounts for. The tracking controller is formulated for the non-linear dynamics of the individual UAVs, and the tracking error bound is computed for this controller when the trajectories satisfy some kinematic constraints. We also augment an existing multi-UAV STL-based trajectory generator in order to generate trajectories that satisfy such constraints. We show that this co-design allows for trajectories that satisfy a given STL specification, and are also dynamically feasible in the sense that they can be tracked with bounded error. The applicability of this approach is demonstrated through simulations of multi-UAV missions.

He Yin, Andrew Packard, Murat Arcak et al.

We present a method for synthesizing controllers to steer trajectories from an initial set to a target set on a finite time horizon. The proposed control synthesis problem is decomposed into two steps. The first step under-approximates the backward reachable set (BRS) from the target set, using level sets of storage functions. The storage function is constructed with an iterative algorithm to maximize the volume of the under-approximated BRS. The second step obtains a control law by solving a pointwise min-norm optimization problem using the pre-computed storage function. A closed-form solution of this min-norm optimization can be computed through the KKT conditions. This control synthesis framework is then extended to uncertain nonlinear systems with parametric uncertainties and L_2 disturbances. The computation algorithm for all cases is derived using sum-of-squares (SOS) programming and the S-procedure. The proposed method is applied to several robotics and aircraft examples.

Samuel Coogan, Murat Arcak

We propose a macroscopic traffic network flow model suitable for analysis as a dynamical system, and we qualitatively analyze equilibrium flows as well as convergence. Flows at a junction are determined by downstream supply of capacity as well as upstream demand of traffic wishing to flow through the junction. This approach is rooted in the celebrated Cell Transmission Model for freeway traffic flow. Unlike related results which rely on certain system cooperativity properties, our model generally does not possess these properties. We show that the lack of cooperativity is in fact a useful feature that allows traffic control methods, such as ramp metering, to be effective. Finally, we leverage the results of the paper to develop a linear program for optimal ramp metering.

John Maidens, Murat Arcak

We present a method of solving the T-optimal design problem for nonlinear dynamical systems using dynamic programming. In contrast with previous dynamic programming formulations, we avoid adding an equation for the dispersion to the system state, allowing for more efficient solutions.