John Leth

Mert Bastug, Mihaly Petreczky, Rafael Wisniewski et al.

Two moment-matching methods for model reduction of linear switched systems (LSSs) are presented. The methods are similar to the Krylov subspace methods used for moment matching for linear systems. The more general one of the two methods, is based on the so called "nice selection" of some vectors in the reachability or observability space of the LSS. The underlying theory is closely related to the (partial) realization theory of LSSs. In this paper, the connection of the methods to the realization theory of LSSs is provided, and algorithms are developed for the purpose of model reduction. Conditions for applicability of the methods for model reduction are stated and finally the results are illustrated on numerical examples.

Deividas Eringis, John Leth, Zheng-Hua Tan et al.

In this paper we derive a Probably Approxilmately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) stochastic dynamical systems with inputs. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. In particular, with the bound derived in this paper relates future average prediction errors with the prediction error generated by the model on the data used for learning. In turn, this allows us to provide finite-sample error bounds for a wide class of learning/system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.

Mert Bastug, Mihaly Petreczky, Rafael Wisniewski et al.

We present a procedure for reducing the number of continuous states of discrete-time linear switched systems, such that the reduced system has the same behavior as the original system for a subset of switching sequences. The proposed method is expected to be useful for abstraction based control synthesis methods for hybrid systems.

Deividas Eringis, John Leth, Zheng-Hua Tan et al.

In this paper we derive a PAC-Bayesian-Like error bound for a class of stochastic dynamical systems with inputs, namely, for linear time-invariant stochastic state-space models (stochastic LTI systems for short). This class of systems is widely used in control engineering and econometrics, in particular, they represent a special case of recurrent neural networks. In this paper we 1) formalize the learning problem for stochastic LTI systems with inputs, 2) derive a PAC-Bayesian-Like error bound for such systems, 3) discuss various consequences of this error bound.

Deividas Eringis, John Leth, Zheng-Hua Tan et al.

In this paper, we derive a PAC-Bayes bound on the generalisation gap, in a supervised time-series setting for a special class of discrete-time non-linear dynamical systems. This class includes stable recurrent neural networks (RNN), and the motivation for this work was its application to RNNs. In order to achieve the results, we impose some stability constraints, on the allowed models. Here, stability is understood in the sense of dynamical systems. For RNNs, these stability conditions can be expressed in terms of conditions on the weights. We assume the processes involved are essentially bounded and the loss functions are Lipschitz. The proposed bound on the generalisation gap depends on the mixing coefficient of the data distribution, and the essential supremum of the data. Furthermore, the bound converges to zero as the dataset size increases. In this paper, we 1) formalize the learning problem, 2) derive a PAC-Bayesian error bound for such systems, 3) discuss various consequences of this error bound, and 4) show an illustrative example, with discussions on computing the proposed bound. Unlike other available bounds the derived bound holds for non i.i.d. data (time-series) and it does not grow with the number of steps of the RNN.

Deividas Eringis, John Leth, Zheng-Hua Tan et al.

In this short article, we showcase the derivation of the optimal (minimum error variance) estimator, when one part of the stochastic LTI system output is not measured but is able to be predicted from the measured system outputs. Similar derivations have been done before but not using state-space representation.

Deividas Eringis, John Leth, Zheng-Hua Tan et al.

In this paper we derive a PAC-Bayesian error bound for autonomous stochastic LTI state-space models. The motivation for deriving such error bounds is that they will allow deriving similar error bounds for more general dynamical systems, including recurrent neural networks. In turn, PACBayesian error bounds are known to be useful for analyzing machine learning algorithms and for deriving new ones.

Mert Bastug, Mihaly Petreczky, Roland Toth et al.

We present a novel algorithm for reducing the state dimension, i.e. order, of linear parameter varying (LPV) discrete-time state-space (SS) models with affine dependence on the scheduling variable. The input-output behavior of the reduced order model approximates that of the original model. In fact, for input and scheduling sequences of a certain length, the input-output behaviors of the reduced and original model coincide. The proposed method can also be interpreted as a reachability and observability reduction (minimization) procedure for LPV-SS representations with affine dependence.