Matthias A. Müller

Yi Yang, Victor G. Lopez, Matthias A. Müller

In this paper, we propose a moving horizon estimation (MHE)-based training method for feedforward neural networks (FNNs) with rectified linear unit (ReLU) activation functions to determine their ideal weights from a control-theoretic perspective. This allows for a rigorous theoretical analysis of the trained network. First, we reformulate the FNN as a dynamical system with the weights as states. Then, we investigate the local observability of such a system. For two-layer FNNs with fixed output weights, we derive a sufficient condition under which the observability rank condition holds, ensuring a locally observable state. We also show that multi-layer FNNs in general fail to satisfy the observability rank condition. Based on this analysis, we develop a persistently exciting (PE) input design method, which renders a state distinguishable from its neighbors. The resulting local observability provides convergence guarantees for the proposed MHE-based training, where only the projection of the state onto the observable subspace is updated using a fixed-length window of input-output data. The effectiveness of the approach is illustrated via numerical examples.

Stefan Wildhagen, Matthias A. Müller, Frank Allgöwer

In this paper, we analyze an economic model predictive control scheme with terminal region and cost, where the system is optimally operated in a certain subset of the state space. The predictive controller operates with a cyclic horizon, which means that starting from an initial length, the horizon is reduced by one at each time step before it is restored to its maximum length again after one cycle. We give performance guarantees for the closed loop, and under a suitable dissipativity condition, establish convergence to the optimal subset. Moreover, we present conditions under which asymptotic stability of the optimal subset can be guaranteed. The results are illustrated in a practical example from the context of Networked Control Systems, which initially motivated the development of the theory presented in this paper.

Tobias M. Wolff, Isabelle Krauss, Victor G. Lopez et al. · tsinghua

In this work, we introduce a sample- and data-based moving horizon estimation framework for linear systems. We perform state estimation in a sample-based fashion in the sense that we assume to have only few, irregular output measurements available. This setting is encountered in applications where measuring is expensive or time-consuming. Furthermore, the state estimation framework does not rely on a standard mathematical model, but on an implicit system representation based on measured data. We prove sample-based practical robust exponential stability of the proposed estimator under mild assumptions. Furthermore, we apply the proposed scheme to estimate the states of a gastrointestinal tract absorption system.

Victor G. Lopez, Matthias A. Müller

In this paper, an off-policy reinforcement learning algorithm is designed to solve the continuous-time LQR problem using only input-state data measured from the system. Different from other algorithms in the literature, we propose the use of a specific persistently exciting input as the exploration signal during the data collection step. We then show that, using this persistently excited data, the solution of the matrix equation in our algorithm is guaranteed to exist and to be unique at every iteration. Convergence of the algorithm to the optimal control input is also proven. Moreover, we formulate the policy evaluation step as the solution of a Sylvester-transpose equation, which increases the efficiency of its solution. Finally, a method to determine a stabilizing policy to initialize the algorithm using only measured data is proposed.

Victor G. Lopez, Matthias A. Müller

The design of direct data-based controllers has become a fundamental part of control theory research in the last few years. In this paper, we consider three classes of data-based state feedback control problems for linear systems. These control problems are such that, besides stabilization, some additional performance requirements must be satisfied. First, we formulate and solve a trajectory-reference control problem, on which desired closed-loop trajectories are known and a controller that allows the system to closely follow those trajectories is computed. Then, the solution of the LQR problem for continuous-time systems is presented. Finally, we consider the case in which the precise position of the desired poles of the closed-loop system is known, and introduce a data-based variant of a robust pole-placement procedure. The applicability of the proposed methods is tested using numerical simulations.

Victor G. Lopez, Matthias A. Müller

This paper considers data-based solutions of linear-quadratic nonzero-sum differential games. Two cases are considered. First, the deterministic game is solved and Nash equilibrium strategies are obtained by using persistently excited data from the multiagent system. Then, a stochastic formulation of the game is considered, where each agent measures a different noisy output signal and state observers must be designed for each player. It is shown that the proposed data-based solutions of these games are equivalent to known model-based procedures. The resulting data-based solutions are validated in a numerical experiment.

Mingzhou Yin, Andrea Iannelli, Seyed Ali Nazari et al.

Extending data-driven algorithms based on Willems' fundamental lemma to stochastic data often requires empirical and customized workarounds. This work presents a unified Bayesian framework for linear systems that provides a systematic and general method for handling stochastic data-driven tasks, including smoothing, prediction, and control, via maximum a posteriori estimation. This framework formulates a unified trajectory estimation problem for the three tasks by specifying different types of trajectory knowledge. Then, a Bayesian problem is solved that optimally combines trajectory knowledge with a data-driven characterization of the trajectory from offline data for correlated input-output uncertainties with elliptical distributions. Under specific conditions, this problem is shown to generalize existing data-driven prediction and control algorithms. Numerical examples demonstrate the performance of the unified approach for all three tasks against other data-driven and system identification approaches.

Seth Siriya, Tobias M. Wolff, Isabelle Krauss et al.

Model-based control techniques have recently been investigated for the recommendation of medication dosages to address thyroid diseases. These techniques often rely on knowledge of internal hormone concentrations that cannot be measured from blood samples. Moreover, the measurable concentrations are typically only obtainable at irregular sampling times. In this work, we empirically verify a notion of sample-based detectability that accounts for irregular sampling of the measurable concentrations on two pituitary-thyroid loop models representing patients with hypo- and hyperthyroidism, respectively, and include the internal concentrations as states. We then implement sample-based moving horizon estimation for the models, and test its performance on virtual patients across a range of sampling schemes. Our study shows robust stability of the estimator across all scenarios, and that more frequent sampling leads to less estimation error in the presence of model uncertainty and misreported dosages.

Gianluca Giacomelli, Victor G. Lopez, Simone Formentin et al.

Data-enabled predictive control (DeePC) has recently attracted attention as a promising approach for controlling systems directly from raw data, without requiring an explicit identification step. However, DeePC has not yet been extended to piecewise affine (PWA) systems, despite their extensive use in the (predictive) control literature and their universal approximation capabilities. To address this gap, in this work, we lay the foundations for data-enabled predictive control of PWA systems, providing: $(i)$ their behavioral characterization; $(ii)$ an extension of Willems' Fundamental Lemma to represent their behavior from raw data; $(iii)$ an analysis of the coherence of DeePC strategies using a linear predictor and shrinkage regularizers; and $(iv)$ a study of the impact of misclassification errors on structuring data for prediction. Our theoretical findings are validated by numerical results on a simple example, emphasizing the need to extend beyond a regularized version of the foundational DeePC framework to design control actions that are both effective and coherent with a PWA system's behavior, thus ensuring the controller's explainability.

Isabelle Krauss, Victor G. Lopez, Matthias A. Müller

In this work, we propose an event-triggered moving horizon estimation (ET-MHE) scheme for the remote state estimation of general nonlinear systems. In the presented method, whenever an event is triggered, a single measurement is transmitted and the nonlinear MHE optimization problem is subsequently solved. If no event is triggered, the current state estimate is updated using an open-loop prediction based on the system dynamics. Moreover, we introduce a novel event-triggering rule under which we demonstrate robust global exponential stability of the ET-MHE scheme, assuming a suitable detectability condition is met. In addition, we show that with the adoption of a varying horizon length, a tighter bound on the estimation error can be achieved. Finally, we validate the effectiveness of the proposed method through two illustrative examples.

Jonas Mair, Lukas Schwenkel, Matthias A. Müller et al.

In this paper, we consider undiscouted infinite-horizon optimal control for deterministic systems with an uncountable state and input space. We specifically address the case when the classic value iteration does not converge. For such systems, we use the Ces`aro mean to define the infinite-horizon optimal control problem and the corresponding infinite-horizon value function. Moreover, for this value function, we introduce the Cesàro value iteration and prove its convergence for the special case of systems with periodic optimal operating behavior. For this instance, we also show that the Cesàro value function recovers the undiscounted infinite-horizon optimal cost, if the latter is well-defined.

Jonas Mair, Lukas Schwenkel, Matthias A. Müller et al.

Operation at steady state is often not optimal when optimizing over an economic cost objective. In many cases, periodic operation yields better performance. Therefore, we derive asymptotic stability guarantees of an economic model predictive control scheme with terminal conditions for systems with optimal periodic operation for a more general setup than existing methods can handle. Moreover, we establish a non-averaged closed-loop performance bound by defining the closed-loop cost via a Cesàro summation instead of ordinary summation. Such a non-averaged performance bound provides new insights for systems with periodic optimal operation.

Julian D. Schiller, Malte Heinrich, Victor G. Lopez et al.

Training recurrent neural networks (RNNs) with standard backpropagation through time (BPTT) can be challenging, especially in the presence of long input sequences. A practical alternative to reduce computational and memory overhead is to perform BPTT repeatedly over shorter segments of the training data set, corresponding to truncated BPTT. In this paper, we examine the training of RNNs when using such a truncated learning approach for time series tasks. Specifically, we establish theoretical bounds on the accuracy and performance loss when optimizing over subsequences instead of the full data sequence. This reveals that the burn-in phase of the RNN is an important tuning knob in its training, with significant impact on the performance guarantees. We validate our theoretical results through experiments on standard benchmarks from the fields of system identification and time series forecasting. In all experiments, we observe a strong influence of the burn-in phase on the training process, and proper tuning can lead to a reduction of the prediction error on the training and test data of more than 60% in some cases.

Mingzhou Yin, Matthias A. Müller

This work investigates data-driven prediction and control of Hammerstein-Wiener systems using physics-informed Gaussian process models. Data-driven prediction algorithms have been developed for structured nonlinear systems based on Willems' fundamental lemma. However, existing frameworks cannot treat output nonlinearities and require a dictionary of basis functions for Hammerstein systems. In this work, an implicit predictor structure is considered, leveraging the multi-step-ahead ARX structure for the linear part of the model. This implicit function is learned by Gaussian process regression with kernel functions designed from Gaussian process priors for the nonlinearities. The linear model parameters are estimated as hyperparameters by assuming a stable spline hyperprior. The implicit Gaussian process model provides explicit output prediction by optimizing selected optimality criteria. The model is also applied to receding horizon control with the expected control cost and chance constraint satisfaction guarantee. Numerical results demonstrate that the proposed prediction and control algorithms are superior to black-box Gaussian process models.