Anastasios Tsiamis

Anastasios Tsiamis, Ingvar Ziemann, Nikolai Matni et al.

This tutorial survey provides an overview of recent non-asymptotic advances in statistical learning theory as relevant to control and system identification. While there has been substantial progress across all areas of control, the theory is most well-developed when it comes to linear system identification and learning for the linear quadratic regulator, which are the focus of this manuscript. From a theoretical perspective, much of the labor underlying these advances has been in adapting tools from modern high-dimensional statistics and learning theory. While highly relevant to control theorists interested in integrating tools from machine learning, the foundational material has not always been easily accessible. To remedy this, we provide a self-contained presentation of the relevant material, outlining all the key ideas and the technical machinery that underpin recent results. We also present a number of open problems and future directions.

Ingvar Ziemann, Anastasios Tsiamis, Bruce Lee et al.

This tutorial serves as an introduction to recently developed non-asymptotic methods in the theory of -- mainly linear -- system identification. We emphasize tools we deem particularly useful for a range of problems in this domain, such as the covering technique, the Hanson-Wright Inequality and the method of self-normalized martingales. We then employ these tools to give streamlined proofs of the performance of various least-squares based estimators for identifying the parameters in autoregressive models. We conclude by sketching out how the ideas presented herein can be extended to certain nonlinear identification problems.

Anastasios Tsiamis, Ingvar Ziemann, Manfred Morari et al.

In this paper, we study the statistical difficulty of learning to control linear systems. We focus on two standard benchmarks, the sample complexity of stabilization, and the regret of the online learning of the Linear Quadratic Regulator (LQR). Prior results state that the statistical difficulty for both benchmarks scales polynomially with the system state dimension up to system-theoretic quantities. However, this does not reveal the whole picture. By utilizing minimax lower bounds for both benchmarks, we prove that there exist non-trivial classes of systems for which learning complexity scales dramatically, i.e. exponentially, with the system dimension. This situation arises in the case of underactuated systems, i.e. systems with fewer inputs than states. Such systems are structurally difficult to control and their system theoretic quantities can scale exponentially with the system dimension dominating learning complexity. Under some additional structural assumptions (bounding systems away from uncontrollability), we provide qualitatively matching upper bounds. We prove that learning complexity can be at most exponential with the controllability index of the system, that is the degree of underactuation.

Aren Karapetyan, Anastasios Tsiamis, Efe C. Balta et al.

The setting of an agent making decisions under uncertainty and under dynamic constraints is common for the fields of optimal control, reinforcement learning, and recently also for online learning. In the online learning setting, the quality of an agent's decision is often quantified by the concept of regret, comparing the performance of the chosen decisions to the best possible ones in hindsight. While regret is a useful performance measure, when dynamical systems are concerned, it is important to also assess the stability of the closed-loop system for a chosen policy. In this work, we show that for linear state feedback policies and linear systems subject to adversarial disturbances, linear regret implies asymptotic stability in both time-varying and time-invariant settings. Conversely, we also show that bounded input bounded state stability and summability of the state transition matrices imply linear regret.

Ingvar Ziemann, Anastasios Tsiamis, Henrik Sandberg et al.

We study stochastic policy gradient methods from the perspective of control-theoretic limitations. Our main result is that ill-conditioned linear systems in the sense of Doyle inevitably lead to noisy gradient estimates. We also give an example of a class of stable systems in which policy gradient methods suffer from the curse of dimensionality. Our results apply to both state feedback and partially observed systems.

Charis Stamouli, Anastasios Tsiamis, George J. Pappas

Layered control is essential for managing complexity in large-scale systems, employing progressively coarser models at higher layers. While significant advances have been made for fully observable systems, the theoretical foundations of layered control under partial observations and stochastic noise remain underexplored. To address this gap, we propose a principled layered control framework for such settings. Given a state estimator at each layer, our approach ensures that the expected output distance between systems at successive layers remains within a priori computable bounds. This is achieved by introducing a novel notion of stochastic simulation functions for partially observed systems. For the class of linear systems with Kalman estimators, we provide a systematic construction of these functions along with the corresponding control design. We demonstrate our framework on two aerial robotic scenarios: an unmanned aerial vehicle and a hexacopter with a camera payload.

Anastasios Vlachos, Anastasios Tsiamis, Aren Karapetyan et al.

In this paper, we consider the problem of predicting unknown targets from data. We propose Online Residual Learning (ORL), a method that combines online adaptation with offline-trained predictions. At a lower level, we employ multiple offline predictions generated before or at the beginning of the prediction horizon. We augment every offline prediction by learning their respective residual error concerning the true target state online, using the recursive least squares algorithm. At a higher level, we treat the augmented lower-level predictors as experts, adopting the Prediction with Expert Advice framework. We utilize an adaptive softmax weighting scheme to form an aggregate prediction and provide guarantees for ORL in terms of regret. We employ ORL to boost performance in the setting of online pedestrian trajectory prediction. Based on data from the Stanford Drone Dataset, we show that ORL can demonstrate best-of-both-worlds performance.

Anastasios Tsiamis, Aren Karapetyan, Yueshan Li et al.

In this paper, we study the problem of online tracking in linear control systems, where the objective is to follow a moving target. Unlike classical tracking control, the target is unknown, non-stationary, and its state is revealed sequentially, thus, fitting the framework of online non-stochastic control. We consider the case of quadratic costs and propose a new algorithm, called predictive linear online tracking (PLOT). The algorithm uses recursive least squares with exponential forgetting to learn a time-varying dynamic model of the target. The learned model is used in the optimal policy under the framework of receding horizon control. We show the dynamic regret of PLOT scales with $\mathcal{O}(\sqrt{TV_T})$, where $V_T$ is the total variation of the target dynamics and $T$ is the time horizon. Unlike prior work, our theoretical results hold for non-stationary targets. We implement PLOT on a real quadrotor and provide open-source software, thus, showcasing one of the first successful applications of online control methods on real hardware.

Nikolaos Bousias, Charalampia Stamouli, Anastasios Tsiamis et al.

Control barrier functions (CBFs) provide a powerful tool for enforcing safety constraints in control systems, but their direct application to complex, high-dimensional dynamics is often challenging. In many settings, safety certificates are more naturally designed for simplified or alternative system models that do not exactly match the dynamics of interest. This paper addresses the problem of transferring safety guarantees between dynamical systems with mismatched dynamics. We propose a transferred control barrier function (tCBF) framework that enables safety constraints defined on one system to be systematically enforced on another system using a simulation function and an explicit margin term. The resulting transferred barrier accounts for model mismatch and induces a safety condition that can be enforced on the target system via a quadratic-program-based safety filter. The proposed approach is general and does not require the two systems to share the same state dimension or dynamics. We demonstrate the effectiveness of the framework on a quadrotor navigation task with the transferred barrier ensuring collision avoidance for the target system, while remaining minimally invasive to a nominal controller. These results highlight the potential of transferred control barrier functions as a general mechanism for enforcing safety across heterogeneous dynamical systems.

Shengling Shi, Anastasios Tsiamis, Bart De Schutter

This work analyzes how the trade-off between the modeling error, the terminal value function error, and the prediction horizon affects the performance of a nominal receding-horizon linear quadratic (LQ) controller. By developing a novel perturbation result of the Riccati difference equation, a novel performance upper bound is obtained and suggests that for many cases, the prediction horizon can be either one or infinity to improve the control performance, depending on the relative difference between the modeling error and the terminal value function error. The result also shows that when an infinite horizon is desired, a finite prediction horizon that is larger than the controllability index can be sufficient for achieving a near-optimal performance, revealing a close relation between the prediction horizon and controllability. The obtained suboptimality performance upper bound is applied to provide novel sample complexity and regret guarantees for nominal receding-horizon LQ controllers in a learning-based setting. We show that an adaptive prediction horizon that increases as a logarithmic function of time is beneficial for regret minimization.

Bruce D. Lee, Anastasios Tsiamis, Nikolai Matni et al.

Learning methods are increasingly used to synthesize controllers from data, yet existing sample-complexity characterizations for continuous control are sharp only in the fully observed setting. This paper studies the partially observed case by deriving information-theoretic lower bounds for learning Linear Quadratic Gaussian (LQG) controllers from offline trajectories generated by a (linear) exploration policy. We prove an $\varepsilon$-local minimax excess-cost lower bound that applies to any algorithm mapping the offline dataset to a stabilizing linear controller. The bound is expressed in terms of the Hessian of the LQG cost with respect to model parameters and the inverse Fisher Information induced by the exploration policy. We further provide system-theoretic characterizations of these objects, enabling transparent construction of hard instances. Instantiating the bound on classical fragile robust-control examples, including variants of the Doyle LQG fragility counterexample and non-minimum-phase systems, demonstrates when fragile robust control problems translate into high sample complexity for learning-enabled control. These results suggest the asymptotic optimality of certainty-equivalent synthesis and motivate the importance of both task-directed experiment design and system co-design for sample-efficient learning in partially observed control.

Anastasios Tsiamis, Mohamed Abdalmoaty, Roy S. Smith et al.

We study non-parametric frequency-domain system identification from a finite-sample perspective. We assume an open loop scenario where the excitation input is periodic and consider the Empirical Transfer Function Estimate (ETFE), where the goal is to estimate the frequency response at certain desired (evenly-spaced) frequencies, given input-output samples. We show that under sub-Gaussian colored noise (in time-domain) and stability assumptions, the ETFE estimates are concentrated around the true values. The error rate is of the order of $\mathcal{O}((d_{\mathrm{u}}+\sqrt{d_{\mathrm{u}}d_{\mathrm{y}}})\sqrt{M/N_{\mathrm{tot}}})$, where $N_{\mathrm{tot}}$ is the total number of samples, $M$ is the number of desired frequencies, and $d_{\mathrm{u}},\,d_{\mathrm{y}}$ are the dimensions of the input and output signals respectively. This rate remains valid for general irrational transfer functions and does not require a finite order state-space representation. By tuning $M$, we obtain a $N_{\mathrm{tot}}^{-1/3}$ finite-sample rate for learning the frequency response over all frequencies in the $ \mathcal{H}_{\infty}$ norm. Our result draws upon an extension of the Hanson-Wright inequality to semi-infinite matrices. We study the finite-sample behavior of ETFE in simulations.

Francesco Micheli, Efe C. Balta, Anastasios Tsiamis et al.

We address the challenge of sequential data-driven decision-making under context distributional uncertainty. This problem arises in numerous real-world scenarios where the learner optimizes black-box objective functions in the presence of uncontrollable contextual variables. We consider the setting where the context distribution is uncertain but known to lie within an ambiguity set defined as a ball in the Wasserstein distance. We propose a novel algorithm for Wasserstein Distributionally Robust Bayesian Optimization that can handle continuous context distributions while maintaining computational tractability. Our theoretical analysis combines recent results in self-normalized concentration in Hilbert spaces and finite-sample bounds for distributionally robust optimization to establish sublinear regret bounds that match state-of-the-art results. Through extensive comparisons with existing approaches on both synthetic and real-world problems, we demonstrate the simplicity, effectiveness, and practical applicability of our proposed method.

Andreea B. Alexandru, Anastasios Tsiamis, George J. Pappas

The least squares problem with L1-regularized regressors, called Lasso, is a widely used approach in optimization problems where sparsity of the regressors is desired. This formulation is fundamental for many applications in signal processing, machine learning and control. As a motivating problem, we investigate a sparse data predictive control problem, run at a cloud service to control a system with unknown model, using L1-regularization to limit the behavior complexity. The input-output data collected for the system is privacy-sensitive, hence, we design a privacy-preserving solution using homomorphically encrypted data. The main challenges are the non-smoothness of the L1-norm, which is difficult to evaluate on encrypted data, as well as the iterative nature of the Lasso problem. We use a distributed ADMM formulation that enables us to exchange substantial local computation for little communication between multiple servers. We first give an encrypted multi-party protocol for solving the distributed Lasso problem, by approximating the non-smooth part with a Chebyshev polynomial, evaluating it on encrypted data, and using a more cost effective distributed bootstrapping operation. For the example of data predictive control, we prefer a non-homogeneous splitting of the data for better convergence. We give an encrypted multi-party protocol for this non-homogeneous splitting of the Lasso problem to a non-homogeneous set of servers: one powerful server and a few less powerful devices, added for security reasons. Finally, we provide numerical results for our proposed solutions.

Anastasios Tsiamis, George J. Pappas

In this paper, we investigate when system identification is statistically easy or hard, in the finite sample regime. Statistically easy to learn linear system classes have sample complexity that is polynomial with the system dimension. Most prior research in the finite sample regime falls in this category, focusing on systems that are directly excited by process noise. Statistically hard to learn linear system classes have worst-case sample complexity that is at least exponential with the system dimension, regardless of the identification algorithm. Using tools from minimax theory, we show that classes of linear systems can be hard to learn. Such classes include, for example, under-actuated or under-excited systems with weak coupling among the states. Having classified some systems as easy or hard to learn, a natural question arises as to what system properties fundamentally affect the hardness of system identifiability. Towards this direction, we characterize how the controllability index of linear systems affects the sample complexity of identification. More specifically, we show that the sample complexity of robustly controllable linear systems is upper bounded by an exponential function of the controllability index. This implies that identification is easy for classes of linear systems with small controllability index and potentially hard if the controllability index is large. Our analysis is based on recent statistical tools for finite sample analysis of system identification as well as a novel lower bound that relates controllability index with the least singular value of the controllability Gramian.

Nikos Tsilivis, Anastasios Tsiamis, Petros Maragos

In this work, we study the problem of finding approximate, with minimum support set, solutions to matrix max-plus equations, which we call sparse approximate solutions. We show how one can obtain such solutions efficiently and in polynomial time for any $\ell_p$ approximation error. Based on these results, we propose a novel method for piecewise-linear fitting of convex multivariate functions, with optimality guarantees for the model parameters and an approximately minimum number of affine regions.

Andreea B. Alexandru, Anastasios Tsiamis, George J. Pappas

We provide an efficient and private solution to the problem of encryption-aware data-driven control. We investigate a Control as a Service scenario, where a client employs a specialized outsourced control solution from a service provider. The privacy-sensitive model parameters of the client's system are either not available or variable. Hence, we require the service provider to perform data-driven control in a privacy-preserving manner on the input-output data samples from the client. To this end, we co-design the control scheme with respect to both control performance and privacy specifications. First, we formulate our control algorithm based on recent results from the behavioral framework, and we prove closeness between the classical formulation and our formulation that accounts for noise and precision errors arising from encryption. Second, we use a state-of-the-art leveled homomorphic encryption scheme to enable the service provider to perform high complexity computations on the client's encrypted data, ensuring privacy. Finally, we streamline our solution by exploiting the rich structure of data, and meticulously employing ciphertext batching and rearranging operations to enable parallelization. This solution achieves more than twofold runtime and memory improvements compared to our prior work.

Anastasios Tsiamis, George Pappas

In this paper, we consider the problem of predicting observations generated online by an unknown, partially observed linear system, which is driven by stochastic noise. For such systems the optimal predictor in the mean square sense is the celebrated Kalman filter, which can be explicitly computed when the system model is known. When the system model is unknown, we have to learn how to predict observations online based on finite data, suffering possibly a non-zero regret with respect to the Kalman filter's prediction. We show that it is possible to achieve a regret of the order of $\mathrm{poly}\log(N)$ with high probability, where $N$ is the number of observations collected. Our work is the first to provide logarithmic regret guarantees for the widely used Kalman filter. This is achieved using an online least-squares algorithm, which exploits the approximately linear relation between future observations and past observations. The regret analysis is based on the stability properties of the Kalman filter, recent statistical tools for finite sample analysis of system identification, and classical results for the analysis of least-squares algorithms for time series. Our regret analysis can also be applied for state prediction of the hidden state, in the case of unknown noise statistics but known state-space basis. A fundamental technical contribution is that our bounds hold even for the class of non-explosive systems, which includes the class of marginally stable systems, which was an open problem for the case of online prediction under stochastic noise.

Anastasios Tsiamis, Nikolai Matni, George J. Pappas

In this paper, we consider the task of designing a Kalman Filter (KF) for an unknown and partially observed autonomous linear time invariant system driven by process and sensor noise. To do so, we propose studying the following two step process: first, using system identification tools rooted in subspace methods, we obtain coarse finite-data estimates of the state-space parameters and Kalman gain describing the autonomous system; and second, we use these approximate parameters to design a filter which produces estimates of the system state. We show that when the system identification step produces sufficiently accurate estimates, or when the underlying true KF is sufficiently robust, that a Certainty Equivalent (CE) KF, i.e., one designed using the estimated parameters directly, enjoys provable sub-optimality guarantees. We further show that when these conditions fail, and in particular, when the CE KF is marginally stable (i.e., has eigenvalues very close to the unit circle), that imposing additional robustness constraints on the filter leads to similar sub-optimality guarantees. We further show that with high probability, both the CE and robust filters have mean prediction error bounded by $\tilde O(1/\sqrt{N})$, where $N$ is the number of data points collected in the system identification step. To the best of our knowledge, these are the first end-to-end sample complexity bounds for the Kalman Filtering of an unknown system.

Anastasios Tsiamis, George J. Pappas

In this paper, we analyze the finite sample complexity of stochastic system identification using modern tools from machine learning and statistics. An unknown discrete-time linear system evolves over time under Gaussian noise without external inputs. The objective is to recover the system parameters as well as the Kalman filter gain, given a single trajectory of output measurements over a finite horizon of length $N$. Based on a subspace identification algorithm and a finite number of $N$ output samples, we provide non-asymptotic high-probability upper bounds for the system parameter estimation errors. Our analysis uses recent results from random matrix theory, self-normalized martingales and SVD robustness, in order to show that with high probability the estimation errors decrease with a rate of $1/\sqrt{N}$. Our non-asymptotic bounds not only agree with classical asymptotic results, but are also valid even when the system is marginally stable.

Anastasios Tsiamis, Konstantinos Gatsis, George Pappas

This paper studies the problem of remote state estimation in the presence of a passive eavesdropper. A sensor measures a linear plant's state and transmits it to an authorized user over a packet-dropping channel, which is susceptible to eavesdropping. Our goal is to design a coding scheme such that the eavesdropper cannot infer the plant's current state, while the user successfully decodes the sent messages. We employ a novel class of codes, termed State-Secrecy Codes, which are fast and efficient for dynamical systems. They apply linear time-varying transformations to the current and past states received by the user. In this way, they force the eavesdropper's information matrix to decrease with asymptotically the same rate as in the open-loop prediction case, i.e. when the eavesdropper misses all messages. As a result, the eavesdropper's minimum mean square error (mmse) for the unstable states grows unbounded, while the respective error for the stable states converges to the open-loop prediction one. These secrecy guarantees are achieved under minimal conditions, which require that, at least once, the user receives the corresponding packet while the eavesdropper fails to intercept it. Meanwhile, the user's estimation performance remains optimal. The theoretical results are illustrated in simulations.

Anastasios Tsiamis, Konstantinos Gatsis, George J. Pappas

In this paper, we study the problem of remote state estimation, in the presence of a passive eavesdropper. An authorized user estimates the state of an unstable linear plant, based on the packets received from a sensor, while the packets may also be intercepted by the eavesdropper. Our goal is to design a coding scheme at the sensor, which encodes the state information, in order to impair the eavesdropper's estimation performance, while enabling the user to successfully decode the sent messages. We introduce a novel class of codes, termed State-Secrecy Codes, which use acknowledgment signals from the user and apply linear time-varying transformations to the current and previously received states. By exploiting the properties of the system's process noise, the channel physical model and the dynamics, these codes manage to be fast, efficient and, thus, suitable for real-time dynamical systems. We prove that under minimal conditions, State-Secrecy Codes achieve perfect secrecy, namely the eavesdropper's estimation error grows unbounded almost surely, while the user's estimation performance is optimal. These conditions only require that at least once, the user receives the corresponding packet while the eavesdropper fails to intercept it. Even one occurrence of this event renders the eavesdropper's error unbounded with asymptotically optimal rate of increase. State-Secrecy Codes are provided and studied for two cases, i) when direct state measurements are available, and ii) when we only have output measurements. The theoretical results are illustrated in simulations.

Anastasios Tsiamis, Konstantinos Gatsis, George J. Pappas

We study the problem of remote state estimation, in the presence of an eavesdropper. An authorized user estimates the state of a linear plant, based on the data received from a sensor, while the data may also be intercepted by the eavesdropper. To maintain confidentiality with respect to state, we introduce a novel control-theoretic definition of perfect secrecy requiring that the user's expected error remains bounded while the eavesdropper's expected error grows unbounded. We propose a secrecy mechanism which guarantees perfect secrecy by randomly withholding sensor information, under the condition that the user's packet reception rate is larger than the eavesdropper's interception rate. Given this mechanism, we also explore the tradeoff between user's utility and confidentiality with respect to the eavesdropper, via an optimization problem. Finally, some examples are studied to provide insights about this tradeoff.