Konstantinos Gatsis

Vasileios Tzoumas, Konstantinos Gatsis, Ali Jadbabaie et al.

In this paper, we focus on applications in machine learning, optimization, and control that call for the resilient selection of a few elements, e.g. features, sensors, or leaders, against a number of adversarial denial-of-service attacks or failures. In general, such resilient optimization problems are hard, and cannot be solved exactly in polynomial time, even though they often involve objective functions that are monotone and submodular. Notwithstanding, in this paper we provide the first scalable, curvature-dependent algorithm for their approximate solution, that is valid for any number of attacks or failures, and which, for functions with low curvature, guarantees superior approximation performance. Notably, the curvature has been known to tighten approximations for several non-resilient maximization problems, yet its effect on resilient maximization had hitherto been unknown. We complement our theoretical analyses with supporting empirical evaluations.

Mark Eisen, Mohammad M. Rashid, Konstantinos Gatsis et al.

We consider the problem of allocating radio resources over wireless communication links to control a series of independent wireless control systems. Low-latency transmissions are necessary in enabling time-sensitive control systems to operate over wireless links with high reliability. Achieving fast data rates over wireless links thus comes at the cost of reliability in the form of high packet error rates compared to wired links due to channel noise and interference. However, the effect of the communication link errors on the control system performance depends dynamically on the control system state. We propose a novel control-communication co-design approach to the low-latency resource allocation problem. We incorporate control and channel state information to make scheduling decisions over time on frequency, bandwidth and data rates across the next-generation Wi-Fi based wireless communication links that close the control loops. Control systems that are closer to instability or further from a desired range in a given control cycle are given higher packet delivery rate targets to meet. Rather than a simple priority ranking, we derive precise packet error rate targets for each system needed to satisfy stability targets and make scheduling decisions to meet such targets while reducing total transmission time. The resulting Control-Aware Low Latency Scheduling (CALLS) method is tested in numerous simulation experiments that demonstrate its effectiveness in meeting control-based goals under tight latency constraints relative to control-agnostic scheduling.

Konstantinos Gatsis, Hamed Hassani, George J. Pappas

The emerging interest in low-latency high-reliability applications, such as connected vehicles, necessitates a new abstraction between communication and control. Thanks to advances in cyber-physical systems over the past decades, we understand this interface for classical bit-rate models of channels as well as packet-loss-type channels. This work proposes a new abstraction characterized as a tradeoff curve between latency, reliability and rate. Our aim is to understand: Do we (control engineers) prefer faster but less reliable communications (with shorter codes), or slower but more reliable communications (with longer codes)? In this paper we examine the tradeoffs between latency and reliability for the problem of estimating dynamical systems over communication channels. Employing different latency-reliability curves derived from practical coding schemes, we develop a co-design methodology, i.e., select the code length depending on the system dynamics to optimize system performance.

Liqun Zhao, Konstantinos Gatsis, Antonis Papachristodoulou

Reinforcement learning (RL) has demonstrated impressive performance in various areas such as video games and robotics. However, ensuring safety and stability, which are two critical properties from a control perspective, remains a significant challenge when using RL to control real-world systems. In this paper, we first provide definitions of safety and stability for the RL system, and then combine the control barrier function (CBF) and control Lyapunov function (CLF) methods with the actor-critic method in RL to propose a Barrier-Lyapunov Actor-Critic (BLAC) framework which helps maintain the aforementioned safety and stability for the system. In this framework, CBF constraints for safety and CLF constraint for stability are constructed based on the data sampled from the replay buffer, and the augmented Lagrangian method is used to update the parameters of the RL-based controller. Furthermore, an additional backup controller is introduced in case the RL-based controller cannot provide valid control signals when safety and stability constraints cannot be satisfied simultaneously. Simulation results show that this framework yields a controller that can help the system approach the desired state and cause fewer violations of safety constraints compared to baseline algorithms.

Keyan Miao, Konstantinos Gatsis

Relying on recent research results on Neural ODEs, this paper presents a methodology for the design of state observers for nonlinear systems based on Neural ODEs, learning Luenberger-like observers and their nonlinear extension (Kazantzis-Kravaris-Luenberger (KKL) observers) for systems with partially-known nonlinear dynamics and fully unknown nonlinear dynamics, respectively. In particular, for tuneable KKL observers, the relationship between the design of the observer and its trade-off between convergence speed and robustness is analysed and used as a basis for improving the robustness of the learning-based observer in training. We illustrate the advantages of this approach in numerical simulations.

Ondrej Bajgar, Alessandro Abate, Konstantinos Gatsis et al.

The goal of Bayesian inverse reinforcement learning (IRL) is recovering a posterior distribution over reward functions using a set of demonstrations from an expert optimizing for a reward unknown to the learner. The resulting posterior over rewards can then be used to synthesize an apprentice policy that performs well on the same or a similar task. A key challenge in Bayesian IRL is bridging the computational gap between the hypothesis space of possible rewards and the likelihood, often defined in terms of Q values: vanilla Bayesian IRL needs to solve the costly forward planning problem - going from rewards to the Q values - at every step of the algorithm, which may need to be done thousands of times. We propose to solve this by a simple change: instead of focusing on primarily sampling in the space of rewards, we can focus on primarily working in the space of Q-values, since the computation required to go from Q-values to reward is radically cheaper. Furthermore, this reversion of the computation makes it easy to compute the gradient allowing efficient sampling using Hamiltonian Monte Carlo. We propose ValueWalk - a new Markov chain Monte Carlo method based on this insight - and illustrate its advantages on several tasks.

Liqun Zhao, Keyan Miao, Konstantinos Gatsis et al.

Reinforcement learning (RL) excels in applications such as video games, but ensuring safety as well as the ability to achieve the specified goals remains challenging when using RL for real-world problems, such as human-aligned tasks where human safety is paramount. This paper provides safety and stability definitions for such human-aligned tasks, and then proposes an algorithm that leverages neural ordinary differential equations (NODEs) to predict human and robot movements and integrates the control barrier function (CBF) and control Lyapunov function (CLF) with the actor-critic method to help to maintain the safety and stability for human-aligned tasks. Simulation results show that the algorithm helps the controlled robot to reach the desired goal state with fewer safety violations and better sample efficiency compared to other methods in a human-aligned task.

Ondrej Bajgar, Dewi S. W. Gould, Jonathon Liu et al.

As AI systems become increasingly autonomous, reliably aligning their decision-making with human preferences is essential. Inverse reinforcement learning (IRL) offers a promising approach to infer preferences from demonstrations. These preferences can then be used to produce an apprentice policy that performs well on the demonstrated task. However, in domains like autonomous driving or robotics, where errors can have serious consequences, we need not just good average performance but reliable policies with formal guarantees -- yet obtaining sufficient human demonstrations for reliability guarantees can be costly. Active IRL addresses this challenge by strategically selecting the most informative scenarios for human demonstration. We introduce PAC-EIG, an information-theoretic acquisition function that directly targets probably-approximately-correct (PAC) guarantees for the learned policy -- providing the first such theoretical guarantee for active IRL with noisy expert demonstrations. Our method maximises information gain about the regret of the apprentice policy, efficiently identifying states requiring further demonstration. We also present Reward-EIG as an alternative when learning the reward itself is the primary objective. Focusing on finite state-action spaces, we prove convergence bounds, illustrate failure modes of prior heuristic methods, and demonstrate our method's advantages experimentally.

Konstantinos Gatsis

Modern cyber-physical architectures use data collected from systems at different physical locations to learn appropriate behaviors and adapt to uncertain environments. However, an important challenge arises as communication exchanges at the edge of networked systems are costly due to limited resources. This paper considers a setup where multiple agents need to communicate efficiently in order to jointly solve a reinforcement learning problem over time-series data collected in a distributed manner. This is posed as learning an approximate value function over a communication network. An algorithm for achieving communication efficiency is proposed, supported with theoretical guarantees, practical implementations, and numerical evaluations. The approach is based on the idea of communicating only when sufficiently informative data is collected.

Konstantinos Gatsis

A key functionality of emerging connected autonomous systems such as smart cities, smart transportation systems, and the industrial Internet-of-Things, is the ability to process and learn from data collected at different physical locations. This is increasingly attracting attention under the terms of distributed learning and federated learning. However, in connected autonomous systems, data transfer takes place over communication networks with often limited resources. This paper examines algorithms for communication-efficient learning for linear regression tasks by exploiting the informativeness of the data. The developed algorithms enable a tradeoff between communication and learning with theoretical performance guarantees and efficient practical implementations.

Konstantinos Gatsis

A key functionality of emerging connected autonomous systems such as smart transportation systems, smart cities, and the industrial Internet-of-Things, is the ability to process and learn from data collected at different physical locations. This is increasingly attracting attention under the terms of distributed learning and federated learning. However, in this setup data transfer takes place over communication resources that are shared among many users and tasks or subject to capacity constraints. This paper examines algorithms for efficiently allocating resources to linear regression tasks by exploiting the informativeness of the data. The algorithms developed enable adaptive scheduling of learning tasks with reliable performance guarantees.

Konstantinos Gatsis, George J. Pappas

Recent control trends are increasingly relying on communication networks and wireless channels to close the loop for Internet-of-Things applications. Traditionally these approaches are model-based, i.e., assuming a network or channel model they are focused on stability analysis and appropriate controller designs. However the availability of such wireless channel modeling is fundamentally challenging in practice as channels are typically unknown a priori and only available through data samples. In this work we aim to develop algorithms that rely on channel sample data to determine the stability and performance of networked control tasks. In this regard our work is the first to characterize the amount of channel modeling that is required to answer such a question. Specifically we examine how many channel data samples are required in order to answer with high confidence whether a given networked control system is stable or not. This analysis is based on the notion of sample complexity from the learning literature and is facilitated by concentration inequalities. Moreover we establish a direct relation between the sample complexity and the networked system stability margin, i.e., the underlying packet success rate of the channel and the spectral radius of the dynamics of the control system. This illustrates that it becomes impractical to verify stability under a large range of plant and channel configurations. We validate our theoretical results in numerical simulations.

Andreea B. Alexandru, Konstantinos Gatsis, Yasser Shoukry et al.

The development of large-scale distributed control systems has led to the outsourcing of costly computations to cloud-computing platforms, as well as to concerns about privacy of the collected sensitive data. This paper develops a cloud-based protocol for a quadratic optimization problem involving multiple parties, each holding information it seeks to maintain private. The protocol is based on the projected gradient ascent on the Lagrange dual problem and exploits partially homomorphic encryption and secure multi-party computation techniques. Using formal cryptographic definitions of indistinguishability, the protocol is shown to achieve computational privacy, i.e., there is no computationally efficient algorithm that any involved party can employ to obtain private information beyond what can be inferred from the party's inputs and outputs only. In order to reduce the communication complexity of the proposed protocol, we introduced a variant that achieves this objective at the expense of weaker privacy guarantees. We discuss in detail the computational and communication complexity properties of both algorithms theoretically and also through implementations. We conclude the paper with a discussion on computational privacy and other notions of privacy such as the non-unique retrieval of the private information from the protocol outputs.

Anastasios Tsiamis, Konstantinos Gatsis, George Pappas

This paper studies the problem of remote state estimation in the presence of a passive eavesdropper. A sensor measures a linear plant's state and transmits it to an authorized user over a packet-dropping channel, which is susceptible to eavesdropping. Our goal is to design a coding scheme such that the eavesdropper cannot infer the plant's current state, while the user successfully decodes the sent messages. We employ a novel class of codes, termed State-Secrecy Codes, which are fast and efficient for dynamical systems. They apply linear time-varying transformations to the current and past states received by the user. In this way, they force the eavesdropper's information matrix to decrease with asymptotically the same rate as in the open-loop prediction case, i.e. when the eavesdropper misses all messages. As a result, the eavesdropper's minimum mean square error (mmse) for the unstable states grows unbounded, while the respective error for the stable states converges to the open-loop prediction one. These secrecy guarantees are achieved under minimal conditions, which require that, at least once, the user receives the corresponding packet while the eavesdropper fails to intercept it. Meanwhile, the user's estimation performance remains optimal. The theoretical results are illustrated in simulations.

Anastasios Tsiamis, Konstantinos Gatsis, George J. Pappas

In this paper, we study the problem of remote state estimation, in the presence of a passive eavesdropper. An authorized user estimates the state of an unstable linear plant, based on the packets received from a sensor, while the packets may also be intercepted by the eavesdropper. Our goal is to design a coding scheme at the sensor, which encodes the state information, in order to impair the eavesdropper's estimation performance, while enabling the user to successfully decode the sent messages. We introduce a novel class of codes, termed State-Secrecy Codes, which use acknowledgment signals from the user and apply linear time-varying transformations to the current and previously received states. By exploiting the properties of the system's process noise, the channel physical model and the dynamics, these codes manage to be fast, efficient and, thus, suitable for real-time dynamical systems. We prove that under minimal conditions, State-Secrecy Codes achieve perfect secrecy, namely the eavesdropper's estimation error grows unbounded almost surely, while the user's estimation performance is optimal. These conditions only require that at least once, the user receives the corresponding packet while the eavesdropper fails to intercept it. Even one occurrence of this event renders the eavesdropper's error unbounded with asymptotically optimal rate of increase. State-Secrecy Codes are provided and studied for two cases, i) when direct state measurements are available, and ii) when we only have output measurements. The theoretical results are illustrated in simulations.

Anastasios Tsiamis, Konstantinos Gatsis, George J. Pappas

We study the problem of remote state estimation, in the presence of an eavesdropper. An authorized user estimates the state of a linear plant, based on the data received from a sensor, while the data may also be intercepted by the eavesdropper. To maintain confidentiality with respect to state, we introduce a novel control-theoretic definition of perfect secrecy requiring that the user's expected error remains bounded while the eavesdropper's expected error grows unbounded. We propose a secrecy mechanism which guarantees perfect secrecy by randomly withholding sensor information, under the condition that the user's packet reception rate is larger than the eavesdropper's interception rate. Given this mechanism, we also explore the tradeoff between user's utility and confidentiality with respect to the eavesdropper, via an optimization problem. Finally, some examples are studied to provide insights about this tradeoff.