Abolfazl Lavaei

Abolfazl Lavaei, Sadegh Esmaeil Zadeh Soudjani, Rupak Majumdar et al.

This paper is concerned with a compositional approach for constructing abstractions of interconnected discrete-time stochastic control systems. The abstraction framework is based on new notions of so-called stochastic simulation functions, using which one can quantify the distance between original interconnected stochastic control systems and their abstractions in the probabilistic setting. Accordingly, one can leverage the proposed results to perform analysis and synthesis over abstract interconnected systems, and then carry the results over concrete ones. In the first part of the paper, we derive sufficient small-gain type conditions for the compositional quantification of the distance in probability between the interconnection of stochastic control subsystems and that of their abstractions. In the second part of the paper, we focus on the class of discrete-time linear stochastic control systems with independent noises in the abstract and concrete subsystems. For this class of systems, we propose a computational scheme to construct abstractions together with their corresponding stochastic simulation functions. We demonstrate the effectiveness of the proposed results by constructing an abstraction (totally 4 dimensions) of the interconnection of four discrete-time linear stochastic control subsystems (together 100 dimensions) in a compositional fashion.

Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

This paper is concerned with a compositional approach for constructing both infinite (reduced-order models) and finite abstractions (a.k.a. finite Markov decision processes (MDPs)) of large-scale interconnected discrete-time stochastic systems. The proposed framework is based on the notion of stochastic simulation functions enabling us to employ an abstract system as a substitution of the original one in the controller design process with guaranteed error bounds. In the first part of the paper, we derive sufficient small-gain type conditions for the compositional quantification of the probabilistic distance between the interconnection of stochastic control subsystems and that of their infinite abstractions. We then construct infinite abstractions together with their corresponding stochastic simulation functions for a particular class of discrete-time nonlinear stochastic control systems. In the second part of the paper, we leverage small-gain type conditions for the compositional construction of finite abstractions. We propose an approach to construct finite MDPs as finite abstractions of concrete models or their reduced-order versions satisfying an incremental input-to-state stability property. We demonstrate the effectiveness of the proposed results by applying our approaches to a fully interconnected network of 20 nonlinear subsystems (totally 100 dimensions). We construct finite MDPs from their reduced-order versions (together 20 dimensions) with guaranteed error bounds on their output trajectories. We also apply the proposed results to a temperature regulation in a circular building and construct compositionally a finite abstraction of a network containing 1000 rooms. We employ the constructed finite abstractions as substitutes to compositionally synthesize policies regulating the temperature in each room for a bounded time horizon.

Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

This paper is concerned with a compositional approach for constructing finite Markov decision processes of interconnected discrete-time stochastic control systems. The proposed approach leverages the interconnection topology and a notion of so-called stochastic storage functions describing joint dissipativity-type properties of subsystems and their abstractions. In the first part of the paper, we derive dissipativity-type compositional conditions for quantifying the error between the interconnection of stochastic control subsystems and that of their abstractions. In the second part of the paper, we propose an approach to construct finite Markov decision processes together with their corresponding stochastic storage functions for classes of discrete-time control systems satisfying some incremental passivablity property. Under this property, one can construct finite Markov decision processes by a suitable discretization of the input and state sets. Moreover, we show that for linear stochastic control systems, the aforementioned property can be readily checked by some matrix inequality. We apply our proposed results to the temperature regulation in a circular building by constructing compositionally a finite Markov decision process of a network containing 200 rooms in which the compositionality condition does not require any constraint on the number or gains of the subsystems. We employ the constructed finite Markov decision process as a substitute to synthesize policies regulating the temperature in each room for a bounded time horizon.

Behrad Samari, Henrik Sandberg, Karl H. Johansson et al.

Model order reduction techniques simplify high-dimensional dynamical systems by deriving lower-dimensional models that retain essential system characteristics. These techniques are crucial for the controller design of complex systems while significantly reducing computational costs. Nevertheless, constructing effective reduced-order models (ROMs) poses considerable challenges, particularly for nonlinear dynamical systems. These challenges are further exacerbated when the actual system model is unavailable, a scenario frequently encountered in real-world applications. In this work, we propose a data-driven framework for constructing ROMs of nonlinear dynamical systems with unknown mathematical models, enabling controller synthesis directly from the resulting ROMs. We establish similarity relations between the output trajectories of the original systems and those of their ROMs by employing the notion of simulation functions (SFs), thereby enabling a formal characterization of their closeness. To achieve this, we collect one set of noise-corrupted input-state data from the system during a finite-time experiment, upon which we propose conditions to construct both ROMs and SFs simultaneously. These conditions are formulated as data-dependent semidefinite programs. We demonstrate that the data-driven ROMs obtained can be employed to synthesize controllers for the original unknown systems, ensuring that they satisfy high-level logic specifications. This is accomplished by first designing controllers for the data-driven ROMs and then translating the results back to the original systems via interface functions, designed directly from the proposed data-dependent conditions. We evaluate the efficacy of our data-driven framework through two case studies, including a challenging benchmark from the model reduction literature: a circuit of chained inverter gates with 20 state variables.

Behrad Samari, Gian Paolo Incremona, Antonella Ferrara et al.

This paper proposes a data-driven approach to designing adaptive suboptimal second-order sliding mode (ASSOSM) controllers for a class of single-input nonlinear systems with partially unknown dynamics, subject to both matched and unmatched disturbances. We first view the system as comprising two coupled dynamics, referred to as the upper and lower dynamics, with the last state serving as a virtual input to the upper dynamics. The proposed control-design methodology then follows a two-stage procedure: (i) designing a virtual state-feedback control law for the upper dynamics and (ii) synthesizing an ASSOSM controller for the full-order system. To this end, we collect noise-corrupted data from the system throughout a finite-time experiment. We then formulate a data-dependent condition, whose feasibility enables the design of a virtual state-feedback control law that renders the closed-loop upper dynamics input-to-state stable with respect to the unmatched disturbance. Building on this virtual state-feedback control law, we subsequently propose a data-driven nonlinear sliding variable, based on which an ASSOSM controller is designed for the full-order system. The state trajectories of the resulting closed-loop system are semiglobally ultimately bounded (S-GUB), with the ultimate bound explicitly depending on the magnitude of the unmatched disturbance. In particular, the control design parameters can be selected for any prescribed bounded set of initial conditions so that the state trajectories of the closed-loop system are S-GUB. Moreover, the effect of the matched disturbance is totally rejected after a finite time. The effectiveness of the proposed method is satisfactorily demonstrated in the simulation.

Mahdieh Zaker, David Angeli, Abolfazl Lavaei

This work focuses on a compositional data-driven approach to verify incremental global asymptotic stability (delta-GAS) over interconnected homogeneous networks of degree one with unknown mathematical dynamics. Our proposed approach leverages the concept of incremental input-to-state stability (delta-ISS) of subsystems, characterized by delta-ISS Lyapunov functions. To implement our data-driven scheme, we initially reframe the delta-ISS Lyapunov conditions as a robust optimization program (ROP). Due to the presence of unknown subsystem dynamics in the ROP constraints, we develop a scenario optimization program (SOP) by gathering data from trajectories of each unknown subsystem. However, since the measured one-step transition data are corrupted by noise with a known bound on its norm, rendering the proposed SOP intractable, we introduce an auxiliary SOP that explicitly accommodates noisy measurements. We solve the auxiliary SOP and construct a delta-ISS Lyapunov function for each subsystem with unknown dynamics. We then leverage a small-gain compositional condition to facilitate the construction of an incremental Lyapunov function for an unknown interconnected network based on the data-driven delta-ISS Lyapunov functions of its individual subsystems, while providing correctness guarantees, incorporating the bound on the noise norm. We demonstrate that our data-driven compositional approach reduces the sample complexity to the subsystem level. To validate the effectiveness of our approach, we apply it to an unknown controlled physical nonlinear homogeneous network of degree one, comprising 10000 subsystems. By gathering noisy data from each unknown subsystem, we demonstrate that the interconnected network is delta-GAS with a correctness guarantee.

Behrad Samari, Henrik Sandberg, Karl H. Johansson et al.

This paper develops a direct data-driven framework for constructing reduced-order models (ROMs) of discrete-time linear dynamical systems with unknown dynamics and process disturbances. The proposed scheme enables controller synthesis on the ROM and its refinement to the original system by an interface function designed using noisy data. To achieve this, the notion of simulation functions (SFs) is employed to establish a formal relation between the original system and its ROM, yielding a quantitative bound on the mismatch between their output trajectories. To construct such relations and interface functions, we rely on data collected from the unknown system. In particular, using noise-corrupted input-state data gathered along a single trajectory of the system, and without identifying the original dynamics, we propose data-dependent conditions, cast as a semidefinite program, for the simultaneous construction of ROMs, SFs, and interface functions. Through a case study, we demonstrate that data-driven controller synthesis on the ROM, combined with controller refinement via the interface function, enables the enforcement of complex specifications beyond stability.

Mahdieh Zaker, Amy Nejati, Abolfazl Lavaei

Infinite networks are complex interconnected systems comprising a countably infinite number of subsystems, for which no fixed upper bound on the number of participating subsystems is specified a priori since it may vary over time as agents join or leave (e.g., vehicles in traffic). In such scenarios, the presence of infinitely many subsystems within the network renders the existing analysis frameworks tailored for finite networks inapplicable to infinite ones. This paper is concerned with offering a data-driven approach, within a compositional framework, for the safety certification of infinite networks with both unknown mathematical models and unknown interconnection topologies. Given the immense computational complexity stemming from the extensive dimension of infinite networks, our approach capitalizes on the joint dissipativity-type properties of subsystems, characterized by storage certificates. We introduce innovative compositional data-driven conditions to construct a barrier certificate for the infinite network leveraging storage certificates of its unknown subsystems derived from data, while offering correctness guarantees for network safety. We demonstrate that our compositional data-driven reasoning eliminates the requirement for checking the traditional dissipativity condition, which typically mandates precise knowledge of the interconnection topology. We illustrate our data-driven results on two physical infinite networks with unknown models and interconnection topologies.

Ali Aminzadeh, MohammadHossein Ashoori, Amy Nejati et al.

This paper develops a physics-informed scenario approach for safety verification of nonlinear systems using barrier certificates (BCs) to ensure that system trajectories remain within safe regions over an infinite time horizon. Designing BCs often relies on an accurate dynamics model; however, such models are often imprecise due to the model complexity involved, particularly when dealing with highly nonlinear systems. In such cases, while scenario approaches effectively address the safety problem using collected data to construct a guaranteed BC for the unknown dynamical system, they often require solving an optimization problem with substantial amounts of data. To address this, we propose a physics-informed scenario approach that selects data samples such that the outputs of the physics-based model and the observed data are sufficiently close. This approach guides the scenario optimization process to eliminate redundant samples and potentially reduce the required dataset size. We validate our approach through three case studies, showcasing its practical application in reducing the required data.

MohammadHossein Ashoori, Ali Aminzadeh, Amy Nejati et al.

This work addresses the critical challenge of guaranteeing safety for complex dynamical systems where precise mathematical models are uncertain and data measurements are corrupted by noise. We develop a physics-guided, direct data-driven framework for synthesizing robust safety controllers for discrete-time nonlinear polynomial systems that are subject to unknown-but-bounded disturbances. To do so, we introduce a notion of safety through robust control barrier certificates, which ensure avoidance of unsafe regions, offering a less conservative alternative to existing methods based on robust invariant sets. To achieve data efficiency, we further integrate physical information, formulated as quadratic constraints on system and control matrices, with observed noisy data. This integration drastically reduces data requirements, enabling robust safety analysis with significantly shorter trajectories compared to purely data-driven methods. The proposed synthesis procedure is formulated as a sum-of-squares optimization program that systematically designs the barrier and its associated controller by leveraging both collected data and underlying physical laws. The efficacy of our framework is demonstrated on three benchmark systems, confirming its ability to offer robust safety guarantees with reduced data demands.

Mahdieh Zaker, Andrii Mironchenko, Amy Nejati et al.

This paper develops a direct data-driven framework for infinite networks with unknown nonlinear polynomial subsystems, enabling the synthesis of controllers that ensure the entire network is uniformly globally asymptotically stable (UGAS). To address scalability challenges arising from high dimensionality, we develop a data-driven approach to construct an input-to-state stable (ISS) Lyapunov function and its corresponding controller for each unknown subsystem using only a single set of noise-corrupted input-state trajectories collected from that subsystem. Once each subsystem admits a data-driven ISS Lyapunov function, we leverage a compositional small-gain framework for infinite-dimensional spaces to construct a global control Lyapunov function and its associated controller, thereby ensuring UGAS of the entire infinite network. The effectiveness of the proposed data-driven approach is demonstrated through three case studies, including infinite networks of spacecraft, Lorenz chaotic systems, and an academic example with a state-dependent control input matrix.

Ben Wooding, Hongchao Zhang, Taylor T. Johnson et al.

While conventional (k=1) discrete-time barrier certificate conditions impose strict safety constraints by requiring the function to be non-increasing at every step, k-inductive barrier certificates relax this by allowing a temporary increase -- up to k-1 times, each within a threshold $ε$ -- while maintaining overall safety, and improving flexibility. This paper leverages neural networks and constructs k-inductive neural barrier certificates (k-NBCs) for (partially) unknown nonlinear systems. While neural networks offer scalability in the design process, they lack formal guarantees, requiring additional approaches such as counterexample-guided inductive synthesis (CEGIS) with satisfiability modulo theories (SMT) for verification. However, the CEGIS-SMT framework requires knowledge of system dynamics, which is unavailable in practical settings. To address this, we leverage the generalization of the Willems et al.'s fundamental lemma, using a single state trajectory, to construct a data-driven representation of (partially) unknown models for SMT verification without sacrificing accuracy. Additionally, CEGIS-SMT further removes the constraint of restricting barrier certificates to specific function classes, such as sum-of-squares, enabling greater flexibility in their design. We validate our approach on three nonlinear case studies with (partially) unknown dynamics.

Omid Akbarzadeh, MohammadHossein Ashoori, Mohammad H. Mamduhi et al.

This work develops a formal framework for the synthesis of packetized safety controllers for discrete-time polynomial stochastic networked control systems (dt-PSNCS) operating under communication constraints, including uplink delays (plant-to-controller) and downlink packet losses (controller-to-actuator). In this setting, the controller is deployed remotely and exchanges information with the plant over an imperfect wireless communication network. Our proposed approach treats the downlink channel as an erasure channel, with packet losses characterized by an independent Bernoulli process. To systematically manage both uplink delays and downlink packet loss, we first introduce a buffer collocated with the plant that accommodates the packetized safety control (PSC) mechanism. We augment the plant and buffer states into a unified augmented-state representation that accurately captures the system evolution in the presence of communication imperfections. Our proposed framework synthesizes safety controllers based on control barrier certificates (CBCs), providing probabilistic safety guarantees that remain robust in the presence of both communication delays and packet losses. To achieve this, we reformulate the safety constraints as a sum-of-squares (SOS) optimization program, thereby facilitating the systematic construction of CBCs and their corresponding safety controllers. We validate the proposed framework through three (physical) case studies, demonstrating its effectiveness and practical applicability.

Mahdieh Zaker, Andrii Mironchenko, Amy Nejati et al.

We propose a direct data-driven method for controller synthesis of infinite networks composed of unknown linear time-invariant subsystems. Using a single set of noise-corrupted input-state trajectories collected from each subsystem, and provided that certain linear matrix inequalities hold, each subsystem is rendered exponentially input-to-state stable (eISS) by locally constructing an eISS control Lyapunov function together with an exponentially input-to-state stabilizing feedback controller. We then compose these local components under a compositional small-gain condition in infinite-dimensional spaces to obtain a global control Lyapunov function and an associated stabilizing controller, ensuring uniform global exponential stability of the infinite network. The approach is validated on a physical case study with unknown dynamics.

Abolfazl Lavaei, Fabio Somenzi, Sadegh Soudjani et al.

A novel reinforcement learning scheme to synthesize policies for continuous-space Markov decision processes (MDPs) is proposed. This scheme enables one to apply model-free, off-the-shelf reinforcement learning algorithms for finite MDPs to compute optimal strategies for the corresponding continuous-space MDPs without explicitly constructing the finite-state abstraction. The proposed approach is based on abstracting the system with a finite MDP (without constructing it explicitly) with unknown transition probabilities, synthesizing strategies over the abstract MDP, and then mapping the results back over the concrete continuous-space MDP with approximate optimality guarantees. The properties of interest for the system belong to a fragment of linear temporal logic, known as syntactically co-safe linear temporal logic (scLTL), and the synthesis requirement is to maximize the probability of satisfaction within a given bounded time horizon. A key contribution of the paper is to leverage the classical convergence results for reinforcement learning on finite MDPs and provide control strategies maximizing the probability of satisfaction over unknown, continuous-space MDPs while providing probabilistic closeness guarantees. Automata-based reward functions are often sparse; we present a novel potential-based reward shaping technique to produce dense rewards to speed up learning. The effectiveness of the proposed approach is demonstrated by applying it to three physical benchmarks concerning the regulation of a room's temperature, control of a road traffic cell, and of a 7-dimensional nonlinear model of a BMW 320i car.

Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

This paper is concerned with a compositional approach for constructing infinite abstractions of interconnected discrete-time stochastic control systems. The proposed approach uses the interconnection matrix and joint dissipativity-type properties of subsystems and their abstractions described by a new notion of so-called stochastic storage functions. The interconnected abstraction framework is based on new notions of so-called stochastic simulation functions, using which one can quantify the distance between original interconnected stochastic control systems and interconnected abstractions in the probabilistic setting. In the first part of the paper, we derive dissipativity-type compositional reasoning for the quantification of the distance in probability between the interconnection of stochastic control subsystems and that of their abstractions. Moreover, we focus on a class of discrete-time nonlinear stochastic control systems with independent noises in the abstract and concrete subsystems, and propose a computational scheme to construct abstractions together with their corresponding stochastic storage functions. In the second part of the paper, we consider specifications expressed as syntactically co-safe linear temporal logic formulae and show how a synthesized policy for the abstract system can be refined to a policy for the original system while providing guarantee on the probability of satisfaction. We demonstrate the effectiveness of the proposed results by constructing an abstraction (totally 3 dimensions) of the interconnection of three discrete-time nonlinear stochastic control subsystems (together 222 dimensions) in a compositional fashion. We also employ the abstraction as a substitute to synthesize a controller enforcing a syntactically co-safe linear temporal logic specification.