Manuel Mazo

Manuel Mazo, Paulo Tabuada

In recent years we have witnessed a move of the major industrial automation providers into the wireless domain. While most of these companies already offer wireless products for measurement and monitoring purposes, the ultimate goal is to be able to close feedback loops over wireless networks interconnecting sensors, computation devices, and actuators. In this paper we present a decentralized event-triggered implementation, over sensor/actuator networks, of centralized nonlinear controllers. Event-triggered control has been recently proposed as an alternative to the more traditional periodic execution of control tasks. In a typical event-triggered implementation, the control signals are kept constant until the violation of a condition on the state of the plant triggers the re-computation of the control signals. The possibility of reducing the number of re-computations, and thus of transmissions, while guaranteeing desired levels of performance makes event-triggered control very appealing in the context of sensor/actuator networks. In these systems the communication network is a shared resource and event-triggered implementations of control laws offer a flexible way to reduce network utilization. Moreover reducing the number of times that a feedback control law is executed implies a reduction in transmissions and thus a reduction in energy expenditures of battery powered wireless sensor nodes.

Manuel Mazo, Ming Cao

In this paper we propose an approach to the implementation of controllers with decentralized strategies triggering controller updates. We consider set-ups with a central node in charge of the computation of the control commands, and a set of not co-located sensors providing measurements to the controller node. The solution we propose does not require measurements from the sensors to be synchronized in time. The sensors in our proposal provide measurements in an aperiodic way triggered by local conditions. Furthermore, in the proposed implementation (most of) the communication between nodes requires only the exchange of one bit of information (per controller update), which could aid in reducing transmission delays and as a secondary effect result in fewer transmissions being triggered.

Manuel Mazo, Paulo Tabuada

There is an increasing demand for controller design techniques capable of addressing the complex requirements of todays embedded applications. This demand has sparked the interest in symbolic control where lower complexity models of control systems are used to cater for complex specifications given by temporal logics, regular languages, or automata. These specification mechanisms can be regarded as qualitative since they divide the trajectories of the plant into bad trajectories (those that need to be avoided) and good trajectories. However, many applications require also the optimization of quantitative measures of the trajectories retained by the controller, as specified by a cost or utility function. As a first step towards the synthesis of controllers reconciling both qualitative and quantitative specifications, we investigate in this paper the use of symbolic models for time-optimal controller synthesis. We consider systems related by approximate (alternating) simulation relations and show how such relations enable the transfer of time-optimality information between the systems. We then use this insight to synthesize approximately time-optimal controllers for a control system by working with a lower complexity symbolic model. The resulting approximately time-optimal controllers are equipped with upper and lower bounds for the time to reach a target, describing the quality of the controller. The results described in this paper were implemented in the Matlab Toolbox Pessoa which we used to workout several illustrative examples reported in this paper.

Giannis Delimpaltadakis, Morteza Lahijanian, Manuel Mazo et al.

Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic systems for control synthesis. However, due to the absence of algorithms for synthesis over IMDPs with continuous action-spaces, the action-space is assumed discrete a-priori, which is a restrictive assumption for many applications. Motivated by this, we introduce continuous-action IMDPs (caIMDPs), where the bounds on transition probabilities are functions of the action variables, and study value iteration for maximizing expected cumulative rewards. Specifically, we decompose the max-min problem associated to value iteration to $|\mathcal{Q}|$ max problems, where $|\mathcal{Q}|$ is the number of states of the caIMDP. Then, exploiting the simple form of these max problems, we identify cases where value iteration over caIMDPs can be solved efficiently (e.g., with linear or convex programming). We also gain other interesting insights: e.g., in certain cases where the action set $\mathcal{A}$ is a polytope, synthesis over a discrete-action IMDP, where the actions are the vertices of $\mathcal{A}$, is sufficient for optimality. We demonstrate our results on a numerical example. Finally, we include a short discussion on employing caIMDPs as abstractions for control synthesis.

Giannis Delimpaltadakis, Manuel Mazo

In this work, we propose a region-based self-triggered control (STC) scheme for nonlinear systems. The state space is partitioned into a finite number of regions, each of which is associated to a uniform inter-event time. The controller, at each sampling time instant, checks to which region does the current state belong, and correspondingly decides the next sampling time instant. To derive the regions along with their corresponding inter-event times, we use approximations of isochronous manifolds, a notion firstly introduced in [1]. This work addresses some theoretical issues of [1] and proposes an effective computational approach that generates approximations of isochronous manifolds, thus enabling the region-based STC scheme. The efficiency of both our theoretical results and the proposed algorithm are demonstrated through simulation examples.

Anton V. Proskurnikov, Manuel Mazo

Synchronization among autonomous agents via local interactions is one of the benchmark problems in multi-agent control. Whereas synchronization algorithms for identical agents have been thoroughly studied, synchronization of heterogeneous networks still remains a challenging problem. The existing algorithms primarily use the internal model principle, assigning to each agent a local copy of some dynamical system (internal model). Synchronization of heterogeneous agents thus reduces to global synchronization of identical generators and local synchronization between the agents and their internal models. The internal model approach imposes a number of restrictions and leads to sophisticated dynamical (and, in general, nonlinear) controllers. At the same time, passive heterogeneous agents can be synchronized by a very simple linear protocol, which is used for consensus of first-order integrators. A natural question arises whether analogous algorithms are applicable to synchronization of agents that do not satisfy the passivity condition. In this paper, we study the synchronization problem for heterogeneous agents that are not passive but satisfy a weaker input feedforward passivity (IFP) condition. We show that such agents can also be synchronized by a simple linear protocol, provided that the interaction graph is strongly connected and the couplings are sufficiently weak. We demonstrate how stability of cooperative adaptive cruise control algorithms and some microscopic traffic flow models reduce to synchronization of heterogeneous IFP agents.

Andrea Peruffo, Manuel Mazo

We employ the scenario approach to compute probably approximately correct (PAC) bounds on the average inter-sample time (AIST) generated by an unknown PETC system, based on a finite number of samples. We extend the scenario approach to multiclass SVM algorithms in order to construct a PAC map between the concrete, unknown state-space and the inter-sample times. We then build a traffic model applying an $\ell$-complete relation and find, in the underlying graph, the cycles of minimum and maximum average weight: these provide lower and upper bounds on the AIST. Numerical benchmarks show the practical applicability of our method, which is compared against model-based state-of-the-art tools.

Anqi Fu, Manuel Mazo

Asynchronous decentralized event-triggered control (ADETC) is an implementation of controllers characterized by decentralized event generation, asynchronous sampling updates, and dynamic quantization. Combining those elements in ADETC results in a parsimonious transmission of information which makes it suitable for wireless networked implementations. We extend the previous work on ADETC by introducing periodic sampling, denoting our proposal asynchronous decentralized periodic event-triggered control (ADPETC), and study the stability and L2-gain of ADPETC for implementations affected by disturbances. In ADPETC, at each sampling time, quantized measurements from those sensors that triggered a local event are transmitted to a dynamic controller that computes control actions; the quantized control actions are then transmitted to the corresponding actuators only if certain events are also triggered for the corresponding actuator. The developed theory is demonstrated and illustrated via a numerical example.

Anton V. Proskurnikov, Manuel Mazo

Control Lyapunov Functions (CLF) method gives a constructive tool for stabilization of nonlinear systems. To find a CLF, many methods have been proposed in the literature, e.g. backstepping for cascaded systems and sum of squares (SOS) programming for polynomial systems. Dealing with continuous-time systems, the CLF-based controller is also continuous-time, whereas practical implementation on a digital platform requires sampled-time control. In this paper, we show that if the continuous-time controller provides exponential stabilization, then an exponentially stabilizing event-triggered control strategy exists with the convergence rate arbitrarily close to the rate of the continuous-time system.

Majid Zamani, Manuel Mazo, Mahmoud Khaled et al.

The last decade has witnessed significant attention on networked control systems (NCS) due to their ubiquitous presence in industrial applications, and, in the particular case of wireless NCS, because of their architectural flexibility and low installation and maintenance costs. In wireless NCS the communication between sensors, controllers, and actuators is supported by a communication channel that is likely to introduce variable communication delays, packet losses, limited bandwidth, and other practical non-idealities leading to numerous technical challenges. Although stability properties of NCS have been investigated extensively in the literature, results for NCS under more complex and general objectives, and in particular results dealing with verification or controller synthesis for logical specifications, are much more limited. This work investigates how to address such complex objectives by constructively deriving symbolic models of NCS, while encompassing the mentioned network non-idealities. The obtained abstracted (symbolic) models can then be employed to synthesize hybrid controllers enforcing rich logical specifications over the concrete NCS models. Examples of such general specifications include properties expressed as formulae in linear temporal logic (LTL) or as automata on infinite strings. We thus provide a general synthesis framework that can be flexibly adapted to a number of NCS setups. We illustrate the effectiveness of the results over some case studies.

Sokratis Kartakis, Anqi Fu, Manuel Mazo et al.

Energy constraint long-range wireless sensor/ actuator based solutions are theoretically the perfect choice to support the next generation of city-scale cyber-physical systems. Traditional systems adopt periodic control which increases network congestion and actuations while burdens the energy consumption. Recent control theory studies overcome these problems by introducing aperiodic strategies, such as event trigger control. In spite of the potential savings, these strategies assume actuator continuous listening while ignoring the sensing energy costs. In this paper, we fill this gap, by enabling sensing and actuator listening duty-cycling and proposing two innovative MAC protocols for three decentralized event trigger control approaches. A laboratory experimental testbed, which emulates a smart water network, was modelled and extended to evaluate the impact of system parameters and the performance of each approach. Experimental results reveal the predominance of the decentralized event-triggered control against the classic periodic control either in terms of communication or actuation by promising significant system lifetime extension.

Dieky Adzkiya, Manuel Mazo

We discuss the scheduling of a set of networked control systems implemented over a shared communication network. Each control loop is described by a linear-time-invariant (LTI) system with an event-triggered implementation. We assume the network can be used by at most one control loop at any time instant and after each controller update, a pre-defined channel occupancy time elapses before the network is available. In our framework we offer the scheduler two options to avoid conflicts: using the event-triggering mechanism, where the scheduler can choose the triggering coefficient; or forcing controller updates at an earlier pre-defined time. Our objective is avoiding communication conflict while guaranteeing stability of all control loops. We formulate the original scheduling problem as a control synthesis problem over a network of timed game automata (NTGA) with a safety objective. The NTGA is obtained by taking the parallel composition of the timed game automata (TGA) associated with the network and with all control loops. The construction of TGA associated with control loops leverages recent results on the abstraction of timing models of event-triggered LTI systems. In our problem, the safety objective is to avoid that update requests from a control loop happen while the network is in use by another task. We showcase the results in some examples.

Daniel Jarne Ornia, Manuel Mazo

We present an approach to reduce the communication required between agents in a Multi-Agent learning system by exploiting the inherent robustness of the underlying Markov Decision Process. We compute so-called robustness surrogate functions (off-line), that give agents a conservative indication of how far their state measurements can deviate before they need to update other agents in the system. This results in fully distributed decision functions, enabling agents to decide when it is necessary to update others. We derive bounds on the optimality of the resulting systems in terms of the discounted sum of rewards obtained, and show these bounds are a function of the design parameters. Additionally, we extend the results for the case where the robustness surrogate functions are learned from data, and present experimental results demonstrating a significant reduction in communication events between agents.

Daniel Jarne Ornia, Licio Romao, Lewis Hammond et al.

Policy robustness in Reinforcement Learning may not be desirable at any cost: the alterations caused by robustness requirements from otherwise optimal policies should be explainable, quantifiable and formally verifiable. In this work we study how policies can be maximally robust to arbitrary observational noise by analysing how they are altered by this noise through a stochastic linear operator interpretation of the disturbances, and establish connections between robustness and properties of the noise kernel and of the underlying MDPs. Then, we construct sufficient conditions for policy robustness, and propose a robustness-inducing scheme, applicable to any policy gradient algorithm, that formally trades off expected policy utility for robustness through lexicographic optimisation, while preserving convergence and sub-optimality in the policy synthesis.

Rudi Coppola, Hovsep Touloujian, Pierfrancesco Ombrini et al.

Rechargeable lithium-ion (Li-ion) batteries are a ubiquitous element of modern technology. In the last decades, the production and design of such batteries and their adjacent embedded charging and safety protocols, denoted by Battery Management Systems (BMS), has taken central stage. A fundamental challenge to be addressed is the trade-off between the speed of charging and the ageing behavior, resulting in the loss of capacity in the battery cell. We rely on a high-fidelity physics-based battery model and propose an approach to data-driven charging and safety protocol design. Following a Counterexample-Guided Inductive Synthesis scheme, we combine Reinforcement Learning (RL) with recent developments in data-driven formal methods to obtain a hybrid control strategy: RL is used to synthesise the individual controllers, and a data-driven abstraction guides their partitioning into a switched structure, depending on the initial output measurements of the battery. The resulting discrete selection among RL-based controllers, coupled with the continuous battery dynamics, realises a hybrid system. When a design meets the desired criteria, the abstraction provides probabilistic guarantees on the closed-loop performance of the cell.

Sasan Vakili, Manuel Mazo, Peyman Mohajerin Esfahani

This paper presents a Bayesian estimation framework for Wiener models, focusing on learning nonlinear output functions under known linear state dynamics. We derive a closed-form optimal affine estimator for the unknown parameters, characterized by the so-called "dynamic basis statistics" (DBS). Several features of the proposed estimator are studied, including Bayesian unbiasedness, closed-form posterior statistics, error monotonicity in trajectory length, and consistency condition (also known as persistent excitation). In the special case of Fourier basis functions, we demonstrate that the closed-form description is computationally available, as the Fourier DBS enjoys explicit expressions. Furthermore, we identify an inherent inconsistency in the Fourier bases for single-trajectory measurements, regardless of the input excitation. Leveraging the closed-form estimation error, we develop an active learning algorithm synthesizing input signals to minimize estimation error. Numerical experiments validate the efficacy of our approach, showing significant improvements over traditional regularized least-squares methods.

Rudi Coppola, Manuel Mazo

Estimating the expectation of a Bernoulli random variable based on N independent trials is a classical problem in statistics, typically addressed using Binomial Proportion Confidence Intervals (BPCI). In the control systems community, many critical tasks-such as certifying the statistical safety of dynamical systems-can be formulated as BPCI problems. Conformal Prediction (CP), a distribution-free technique for uncertainty quantification, has gained significant attention in recent years and has been applied to various control systems problems, particularly to address uncertainties in learned dynamics or controllers. A variant known as training-conditional CP was recently employed to tackle the problem of safety certification. In this note, we highlight that the use of training-conditional CP in this context does not provide valid safety guarantees. We demonstrate why CP is unsuitable for BPCI problems and argue that traditional BPCI methods are better suited for statistical safety certification.

Daniel Jarne Ornia, Manuel Mazo

We present an approach to reduce the communication of information needed on a Distributed Q-Learning system inspired by Event Triggered Control (ETC) techniques. We consider a baseline scenario of a distributed Q-learning problem on a Markov Decision Process (MDP). Following an event-based approach, N agents explore the MDP and communicate experiences to a central learner only when necessary, which performs updates of the actor Q functions. We design an Event Based distributed Q learning system (EBd-Q), and derive convergence guarantees with respect to a vanilla Q-learning algorithm. We present experimental results showing that event-based communication results in a substantial reduction of data transmission rates in such distributed systems. Additionally, we discuss what effects (desired and undesired) these event-based approaches have on the learning processes studied, and how they can be applied to more complex multi-agent systems.

Steven Adams, Daniel Jarne Ornia, Manuel Mazo

We present a biologically inspired design for swarm foraging based on ant's pheromone deployment, where the swarm is assumed to have very restricted capabilities. The robots do not require global or relative position measurements and the swarm is fully decentralized and needs no infrastructure in place. Additionally, the system only requires one-hop communication over the robot network, we do not make any assumptions about the connectivity of the communication graph and the transmission of information and computation is scalable versus the number of agents. This is done by letting the agents in the swarm act as foragers or as guiding agents (beacons). We present experimental results computed for a swarm of Elisa-3 robots on a simulator, and show how the swarm self-organizes to solve a foraging problem over an unknown environment, converging to trajectories around the shortest path. At last, we discuss the limitations of such a system and propose how the foraging efficiency can be increased.

Daniel Jarne Ornia, Pedro J Zufiria, Manuel Mazo

Collaborative multi-agent robotic systems where agents coordinate by modifying a shared environment often result in undesired dynamical couplings that complicate the analysis and experiments when solving a specific problem or task. Simultaneously, biologically-inspired robotics rely on simplifying agents and increasing their number to obtain more efficient solutions to such problems, drawing similarities with natural processes. In this work we focus on the problem of a biologically-inspired multi-agent system solving collaborative foraging. We show how mean field techniques can be used to re-formulate such a stochastic multi-agent problem into a deterministic autonomous system. This de-couples agent dynamics, enabling the computation of limit behaviours and the analysis of optimality guarantees. Furthermore, we analyse how having finite number of agents affects the performance when compared to the mean field limit and we discuss the implications of such limit approximations in this multi-agent system, which have impact on more general collaborative stochastic problems.

Cees F. Verdier, Manuel Mazo

This paper presents an automatic formal controller synthesis method for nonlinear sampled-data systems with safety and reachability specifications. Fundamentally, the presented method is not restricted to polynomial systems and controllers. We consider periodically switched controllers based on a Control Lyapunov Barrier-like functions. The proposed method utilizes genetic programming to synthesize these functions as well as the controller modes. Correctness of the controller are subsequently verified by means of a Satisfiability Modulo Theories solver. Effectiveness of the proposed methodology is demonstrated on multiple systems.

Ivan S. Zapreev, Cees Verdier, Manuel Mazo

Controller synthesis techniques based on symbolic abstractions appeal by producing correct-by-design controllers, under intricate behavioural constraints. Yet, being relations between abstract states and inputs, such controllers are immense in size, which makes them futile for em- bedded platforms. Control-synthesis tools such as PESSOA, SCOTS, and CoSyMA tackle the problem by storing controllers as binary decision di- agrams (BDDs). However, due to redundantly keeping multiple inputs per-state, the resulting controllers are still too large. In this work, we first show that choosing an optimal controller determinization is an NP- complete problem. Further, we consider the previously known controller determinization technique and discuss its weaknesses. We suggest several new approaches to the problem, based on greedy algorithms, symbolic regression, and (muli-terminal) BDDs. Finally, we empirically compare the techniques and show that some of the new algorithms can produce up to 85% smaller controllers than those obtained with the previous technique.

David Casillas-Perez, Daniel Pizarro, Manuel Mazo et al.

This article is a study about the existence and the uniqueness of solutions of a specific quadratic first-order ODE that frequently appears in multiple reconstruction problems. It is called the \emph{planar-perspective equation} due to the duality with the geometric problem of reconstruction of planar-perspective curves from their modulus. Solutions of the \emph{planar-perspective equation} are related with planar curves parametrized with perspective parametrization due to this geometric interpretation. The article proves the existence of only two local solutions to the \emph{initial value problem} with \emph{regular initial conditions} and a maximum of two analytic solutions with \emph{critical initial conditions}. The article also gives theorems to extend the local definition domain where the existence of both solutions are guaranteed. It introduces the \emph{maximal depth function} as a function that upper-bound all possible solutions of the \emph{planar-perspective equation} and contains all its possible \emph{critical points}. Finally, the article describes the \emph{maximal-depth solution problem} that consists of finding the solution of the referred equation that has maximum the depth and proves its uniqueness. It is an important problem as it does not need initial conditions to obtain the unique solution and its the frequent solution that practical algorithms of the state-of-the-art give.

Arman Sharifi Kolarijani, Manuel Mazo, Tamas Keviczky

In networked control systems, the advent of event-triggering strategies in the sampling process has resulted in the usage reduction of network capacities, such as communication bandwidth. However, the aperiodic nature of sampling periods generated by event-triggering strategies has hindered the schedulability of such networks. In this study, we propose a framework to construct a timed safety automaton that captures the sampling behavior of perturbed LTI systems with an $\mathcal{L}_2$-based triggering mechanisms proposed in the Literature. In this framework, the state-space is partitioned into a finite number of convex polyhedral cones, each cone representing a discrete mode in the abstracted automaton. Adopting techniques from stability analysis of retarded systems accompanied with a polytopic embedding of time, LMI conditions to characterize the sampling interval associated with each region are derived. Then, using reachability analysis, the transitions in the abstracted automaton are derived.