Johan Schoukens

Johan Schoukens, Lennart Ljung

The goal of this article is twofold. Firstly, nonlinear system identification is introduced to a wide audience, guiding practicing engineers and newcomers in the field to a sound solution of their data driven modeling problems for nonlinear dynamic systems. In addition, the article also provides a broad perspective on the topic to researchers that are already familiar with the linear system identification theory, showing the similarities and differences between the linear and nonlinear problem. The reader will be referred to the existing literature for detailed mathematical explanations and formal proofs. Here the focus is on the basic philosophy, giving an intuitive understanding of the problems and the solutions, by making a guided tour along the wide range of user choices in nonlinear system identification. Guidelines will be given in addition to many examples, to reach that goal.

Johan Schoukens, Mark Vaes, Rik Pintelon

This article addresses the following problems: 1) First, a nonlinearity analysis is made looking for the presence of nonlinearities in an early phase of the identification process. The level and the nature of the nonlinearities should be retrieved without a significant increase in the amount of measured data. 2) Next it is studied if it is safe to use a linear system identification approach, even if the presence of nonlinear distortions is detected. The properties of the linear system identification approach under these conditions are studied, and the reliability of the uncertainty bounds is checked. 3) Eventually, tools are provided to check how much can be gained if a nonlinear model were identified instead of a linear model. Addressing these three questions forms the outline of this article. The possibilities and pitfalls of using a linear identification framework in the presence of nonlinear distortions will be discussed and illustrated on lab-scale and industrial examples. In this article, the focus is on nonparametric and parametric black box identification methods, however the results might also be useful for physical modeling methods. Knowing the actual nonlinear distortion level can help to choose the required level of detail that is needed in the physical model. This will strongly influence the modeling effort. Also, in this case, significant time can be saved if it is known from experiments that the system behaves almost linearly. The converse is also true. If the experiments show that some (sub-)systems are highly nonlinear, it helps to focus the physical modeling effort on these critical elements.

Georgios Birpoutsoukis, Anna Marconato, John Lataire et al.

In this paper, the regularization approach introduced recently for nonparametric estimation of linear systems is extended to the estimation of nonlinear systems modelled as Volterra series. The kernels of order higher than one, representing higher dimensional impulse responses in the series, are considered to be realizations of multidimensional Gaussian processes. Based on this, prior information about the structure of the Volterra kernel is introduced via an appropriate penalization term in the least squares cost function. It is shown that the proposed method is able to deliver accurate estimates of the Volterra kernels even in the case of a small amount of data points.

Jean-Philippe Noël, Alireza F. Esfahani, Gaetan Kerschen et al.

Most studies tackling hysteresis identification in the technical literature follow white-box approaches, i.e. they rely on the assumption that measured data obey a specific hysteretic model. Such an assumption may be a hard requirement to handle in real applications, since hysteresis is a highly individualistic nonlinear behaviour. The present paper adopts a black-box approach based on nonlinear state-space models to identify hysteresis dynamics. This approach is shown to provide a general framework to hysteresis identification, featuring flexibility and parsimony of representation. Nonlinear model terms are constructed as a multivariate polynomial in the state variables, and parameter estimation is performed by minimising weighted least-squares cost functions. Technical issues, including the selection of the model order and the polynomial degree, are discussed, and model validation is achieved in both broadband and sine conditions. The study is carried out numerically by exploiting synthetic data generated via the Bouc-Wen equations.

Koen Tiels, Johan Schoukens

Many nonlinear systems can be described by a Wiener-Schetzen model. In this model, the linear dynamics are formulated in terms of orthonormal basis functions (OBFs). The nonlinearity is modeled by a multivariate polynomial. In general, an infinite number of OBFs is needed for an exact representation of the system. This paper considers the approximation of a Wiener system with finite-order infinite impulse response dynamics and a polynomial nonlinearity. We propose to use a limited number of generalized OBFs (GOBFs). The pole locations, needed to construct the GOBFs, are estimated via the best linear approximation of the system. The coefficients of the multivariate polynomial are determined with a linear regression. This paper provides a convergence analysis for the proposed identification scheme. It is shown that the estimated output converges in probability to the exact output. Fast convergence rates, in the order $O_p({N_F}^{-n_{rep}/2})$, can be achieved, with $N_F$ the number of excited frequencies and $n_{rep}$ the number of repetitions of the GOBFs.

Anna Marconato, Maarten Schoukens, Johan Schoukens

In the last years, the success of kernel-based regularisation techniques in solving impulse response modelling tasks has revived the interest on linear system identification. In this work, an alternative perspective on the same problem is introduced. Instead of relying on a Bayesian framework to include assumptions about the system in the definition of the covariance matrix of the parameters, here the prior knowledge is injected at the cost function level. The key idea is to define the regularisation matrix as a filtering operation on the parameters, which allows for a more intuitive formulation of the problem from an engineering point of view. Moreover, this results in a unified framework to model low-pass, band-pass and high-pass systems, and systems with one or more resonances. The proposed filter-based approach outperforms the existing regularisation method based on the TC and DC kernels, as illustrated by means of Monte Carlo simulations on several linear modelling examples.

Rishi Relan, Yousef Firouz, Jean-Marc Timmermans et al.

Lithium ion batteries are attracting significant and growing interest, because their high energy and high power density render them an excellent option for energy storage, particularly in hybrid and electric vehicles. In this brief, a data-driven polynomial nonlinear state-space model is proposed for the operating points at the cusp of linear and nonlinear regimes of the battery's electrical operation, based on the thorough nonparametric frequency domain characterization and quantification of the battery's behavior in terms of its linear and nonlinear behavior at different levels of the state of charge.

Georgios Birpoutsoukis, Péter Zoltán Csurcsia, Johan Schoukens

This paper presents an efficient nonparametric time domain nonlinear system identification method. It is shown how truncated Volterra series models can be efficiently estimated without the need of long, transient-free measurements. The method is a novel extension of the regularization methods that have been developed for impulse response estimates of linear time invariant systems. To avoid the excessive memory needs in case of long measurements or large number of estimated parameters, a practical gradient-based estimation method is also provided, leading to the same numerical results as the proposed Volterra estimation method. Moreover, the transient effects in the simulated output are removed by a special regularization method based on the novel ideas of transient removal for Linear Time-Varying (LTV) systems. Combining the proposed methodologies, the nonparametric Volterra models of the cascaded water tanks benchmark are presented in this paper. The results for different scenarios varying from a simple Finite Impulse Response (FIR) model to a 3rd degree Volterra series with and without transient removal are compared and studied. It is clear that the obtained models capture the system dynamics when tested on a validation dataset, and their performance is comparable with the white-box (physical) models.

Johan Schoukens, Rik Pintelon, Yves Rolain et al.

In this paper we show that it is possible to retrieve structural information about complex block-oriented nonlinear systems, starting from linear approximations of the nonlinear system around different setpoints.The key idea is to monitor the movements of the poles and zeros of the linearized models and to reduce the number of candidate models on the basis of these observations. Besides the well known open loop single branch Wiener-, Hammerstein-, and Wiener-Hammerstein systems, we also cover a number of more general structures like parallel (multi branch) Wiener-Hammerstein models, and closed loop block oriented models, including linear fractional representation (LFR) models.

Jean-Philippe Noël, Johan Schoukens

The present paper deals with the identification of nonlinear mechanical vibrations. A grey-box, or semi-physical, nonlinear state-space representation is introduced, expressing the nonlinear basis functions using a limited number of measured output variables. This representation assumes that the observed nonlinearities are localised in physical space, which is a generic case in mechanics. A two-step identification procedure is derived for the grey-box model parameters, integrating nonlinear subspace initialisation and weighted least-squares optimisation. The complete procedure is applied to an electrical circuit mimicking the behaviour of a single-input, single-output (SISO) nonlinear mechanical system and to a single-input, multiple-output (SIMO) geometrically nonlinear beam structure.

Jan Decuyper, Tim De Troyer, Mark Runacres et al.

The flow-induced vibration of bluff bodies is an important problem of many marine, civil, or mechanical engineers. In the design phase of such structures, it is vital to obtain good predictions of the fluid forces acting on the structure. Current methods rely on computational fluid dynamic simulations (CFD), with a too high computational cost to be effectively used in the design phase or for control applications. Alternative methods use heuristic mathematical models of the fluid forces, but these lack the accuracy (they often assume the system to be linear) or flexibility to be useful over a wide operating range. In this work we show that it is possible to build an accurate, flexible and low-computational-cost mathematical model using nonlinear system identification techniques. This model is data driven: it is trained over a user-defined region of interest using data obtained from experiments or simulations, or both. Here we use a Van der Pol oscillator as well as CFD simulations of an oscillating circular cylinder to generate the training data. Then a discrete-time polynomial nonlinear state-space model is fit to the data. This model relates the oscillation of the cylinder to the force that the fluid exerts on the cylinder. The model is finally validated over a wide range of oscillation frequencies and amplitudes, both inside and outside the so-called lock-in region. We show that forces simulated by the model are in good agreement with the data obtained from CFD.

Alireza Fakhrizadeh Esfahani, Philippe Dreesen, Koen Tiels et al.

Hysteresis is a highly nonlinear phenomenon, showing up in a wide variety of science and engineering problems. The identification of hysteretic systems from input-output data is a challenging task. Recent work on black-box polynomial nonlinear state-space modeling for hysteresis identification has provided promising results, but struggles with a large number of parameters due to the use of multivariate polynomials. This drawback is tackled in the current paper by applying a decoupling approach that results in a more parsimonious representation involving univariate polynomials. This work is carried out numerically on input-output data generated by a Bouc-Wen hysteretic model and follows up on earlier work of the authors. The current article discusses the polynomial decoupling approach and explores the selection of the number of univariate polynomials with the polynomial degree, as well as the connections with neural network modeling. We have found that the presented decoupling approach is able to reduce the number of parameters of the full nonlinear model up to about 50\%, while maintaining a comparable output error level.

Alexander De Cock, Michel Gevers, Johan Schoukens

Optimal input design is an important step of the identification process in order to reduce the model variance. In this work a D-optimal input design method for finite-impulse-response-type nonlinear systems is presented. The optimization of the determinant of the Fisher information matrix is expressed as a convex optimization problem. This problem is then solved using a dispersion-based optimization scheme, which is easy to implement and converges monotonically to the optimal solution. Without constraints, the optimal design cannot be realized as a time sequence. By imposing that the design should lie in the subspace described by a symmetric and non-overlapping set, a realizable design is found. A graph-based method is used in order to find a time sequence that realizes this optimal constrained design. These methods are illustrated on a numerical example of which the results are thoroughly discussed. Additionally the computational speed of the algorithm is compared with the general convex optimizer cvx.

Koen Tiels, Maarten Schoukens, Johan Schoukens

Block-oriented models are often used to model nonlinear systems. These models consist of linear dynamic (L) and nonlinear static (N) sub-blocks. This paper addresses the generation of initial estimates for a Wiener-Hammerstein model (LNL cascade). While it is easy to measure the product of the two linear blocks using a Gaussian excitation and linear identification methods, it is difficult to split the global dynamics over the individual blocks. This paper first proposes a well-designed multisine excitation with pairwise coupled random phases. Next, a modified best linear approximation is estimated on a shifted frequency grid. It is shown that this procedure creates a shift of the input dynamics with a known frequency offset, while the output dynamics do not shift. The resulting transfer function, which has complex coefficients due to the frequency shift, is estimated with a modified frequency domain estimation method. The identified poles and zeros can be assigned to either the input or output dynamics. Once this is done, it is shown in the literature that the remaining initialization problem can be solved much easier than the original one. The method is illustrated on experimental data obtained from the Wiener-Hammerstein benchmark system.

Alessio De Angelis, Johan Schoukens, Keith R. Godfrey et al.

Several issues related to the practical synthesis of ternary sequences with specified spectra are addressed in this paper. Specifically, sequences with harmonic multiples of two and three suppressed are studied, given their relevance when testing and characterizing nonlinear systems. In particular, the effect of non-uniform Digital to Analog Converter (DAC) levels on the spectral properties of the generated signal is analyzed. It is analytically shown that the DAC non-uniform levels result in degraded harmonic suppression performance. Moreover, a new approach is proposed for designing ternary sequences, which is flexible and can be adapted to suit different requirements. The resulting sequences, denoted as randomized constrained sequences, are characterized theoretically by deriving an analytical expression of the power spectral density. Furthermore, they are extensively compared with three synthesis approaches proposed in the literature. The approach is validated by numerical simulations and experimental results, showing the potential to achieve harmonic suppression performance of approximately 100 dB.

Erliang Zhang, Maarten Schoukens, Johan Schoukens

Identification of nonlinear block-oriented models has been extensively studied. The presence of the process noise, more precisely its location in the block-oriented model influences essentially the development of a consistent identification algorithm. The present work is proposed with the aim to localize the process noise in the block-oriented model for accurate nonlinear modeling. To this end, the response of a Wiener-Hammerstein system is theoretically analyzed, the disturbance component in the output, caused by the process noise preceding the static nonlinearity, is shown to be dependent on the input signal. Inspired by such theoretical observation, a simple and new protocol is developed to determine the location of the process noise with respect to the static nonlinearity by using an input signal that is periodic, but nonstationary within one period. In addition, the proposed technique is promising to detect the type of certain static nonlinearity (e.g., dead-zone, saturation). Finally, it is validated on a simulated example and a real-life benchmark.

Alireza Fakhrizadeh Esfahani, Johan Schoukens, Laurent Vanbeylen

Block oriented model structure detection is quite desirable since it helps to imagine the system with real physical elements. In this work we explore experimental methods to detect the internal structure of the system, using a black box approach. Two different strategies are compared and the best combination of these is introduced. The methods are applied on two real systems with a static nonlinear block in the feedback path. The main goal is to excite the system in a way that reduces the total distortion in the measured frequency response functions to have more precise measurements and more reliable decision about the structure of the system.

Maarten Schoukens, Rik Pintelon, Tadeusz P. Dobrowiecki et al.

The Best Linear Approximation (BLA) framework has already proven to be a valuable tool to analyze nonlinear systems and to start the nonlinear modeling process. The existing BLA framework is limited to systems with additive (colored) noise at the output. Such a noise framework is a simplified representation of reality. Process noise can play an important role in many real-life applications. This paper generalizes the Best Linear Approximation framework to account also for the process noise, both for the open-loop and the closed-loop setting, and shows that the most important properties of the existing BLA framework remain valid. The impact of the process noise contributions on the robust BLA estimation method is also analyzed.

Anna Marconato, Maarten Schoukens, Koen Tiels et al.

In this paper, several advanced data-driven nonlinear identification techniques are compared on a specific problem: a simplified glucoregulatory system modeling example. This problem represents a challenge in the development of an artificial pancreas for T1DM treatment, since for this application good nonlinear models are needed to design accurate closed-loop controllers to regulate the glucose level in the blood. Block-oriented as well as state-space models are used to describe both the dynamics and the nonlinear behavior of the insulin-glucose system, and the advantages and drawbacks of each method are pointed out. The obtained nonlinear models are accurate in simulating the patient's behavior, and some of them are also sufficiently simple to be considered in the implementation of a model-based controller to develop the artificial pancreas.

Rishi Relan, Koen Tiels, Anna Marconato et al.

Many real world systems exhibit a quasi linear or weakly nonlinear behavior during normal operation, and a hard saturation effect for high peaks of the input signal. In this paper, a methodology to identify a parsimonious discrete-time nonlinear state space model (NLSS) for the nonlinear dynamical system with relatively short data record is proposed. The capability of the NLSS model structure is demonstrated by introducing two different initialisation schemes, one of them using multivariate polynomials. In addition, a method using first-order information of the multivariate polynomials and tensor decomposition is employed to obtain the parsimonious decoupled representation of the set of multivariate real polynomials estimated during the identification of NLSS model. Finally, the experimental verification of the model structure is done on the cascaded water-benchmark identification problem.

Rishi Relan, Koen Tiels, Jean-Marc Timmermans et al.

Battery short-term electrical impedance behavior varies between linear, linear time-varying, or nonlinear at different operating conditions. Data-based electrical impedance modeling techniques often model the battery as a linear time-invariant system at all operating conditions. In addition, these techniques require extensive and time consuming experimentation. Often due to sensor failures during experiments, constraints in data acquisition hardware, varying operating conditions, and the slow dynamics of the battery, it is not always possible to acquire data in a single experiment. Hence, multiple experiments must be performed. In this letter, a local polynomial approach is proposed to estimate nonparametrically the best linear approximation of the electrical impedance affected by varying levels of nonlinear distortion, from a series of input current and output voltage data subrecords of arbitrary length.

Rishi Relan, Johan Schoukens

Discrete-time models are very convenient to simulate a nonlinear system on a computer. In order to build the discrete-time simulation models for the nonlinear feedback systems (which is a very important class of systems in many applications) described as y(t) = g1(u(t), y(t)), one has to solve at each time step a nonlinear algebraic loop for y(t). If a delay is present in the loop, i.e., y(t) = g2(u(t), y(t -1)), fast recursive simulation models can be developed and the need to solve the nonlinear differential-algebraic equations is removed. In this paper, we use the latter to model the nonlinear feedback system using recursive discrete-time models. Theoretical error bounds for such kind of approximated models are provided in the case of band-limited signals, and furthermore, a measurement methodology is proposed for quantifying and validating the output error bounds experimentally.

Jean-Philippe Noël, Johan Schoukens, Gaetan Kerschen

In the present paper, a flexible and parsimonious model of the vibrations of nonlinear mechanical systems is introduced in the form of state-space equations. It is shown that the nonlinear model terms can be formed using a limited number of output measurements. A two-step identification procedure is derived for this grey-box model, integrating nonlinear subspace initialisation and maximum likelihood optimisation. The complete procedure is demonstrated on the Silverbox benchmark, which is an electrical mimicry of a single-degree-of-freedom mechanical system with one displacement-dependent nonlinearity.

Gabriel Hollander, Philippe Dreesen, Mariya Ishteva et al.

Multivariate polynomials arise in many different disciplines. Representing such a polynomial as a vector of univariate polynomials can offer useful insight, as well as more intuitive understanding. For this, techniques based on tensor methods are known, but these have only been studied in the exact case. In this paper, we generalize an existing method to the noisy case, by introducing a weight factor in the tensor decomposition. Finally, we apply the proposed weighted decoupling algorithm in the domain of system identification, and observe smaller model errors.

Philippe Dreesen, David Westwick, Johan Schoukens et al.

Providing flexibility and user-interpretability in nonlinear system identification can be achieved by means of block-oriented methods. One of such block-oriented system structures is the parallel Wiener-Hammerstein system, which is a sum of Wiener-Hammerstein branches, consisting of static nonlinearities sandwiched between linear dynamical blocks. Parallel Wiener-Hammerstein models have more descriptive power than their single-branch counterparts, but their identification is a non-trivial task that requires tailored system identification methods. In this work, we will tackle the identification problem by performing a tensor decomposition of the Volterra kernels obtained from the nonlinear system. We illustrate how the parallel Wiener-Hammerstein block-structure gives rise to a joint tensor decomposition of the Volterra kernels with block-circulant structured factors. The combination of Volterra kernels and tensor methods is a fruitful way to tackle the parallel Wiener-Hammerstein system identification task. In simulation experiments, we were able to reconstruct very accurately the underlying blocks under noisy conditions.

Kaushik Mahata, Johan Schoukens

We present a closed form expression for the information matrix associated with the Wiener model identification problem under the assumption that the input signal is a stationary Gaussian process. This expression holds under quite generic assumptions. We allow the linear sub-system to have a rational transfer function of arbitrary order, and the static nonlinearity to be a polynomial of arbitrary degree. We also present a simple expression for the determinant of the information matrix. The expressions presented herein has been used for optimal experiment design for Wiener model identification.

Sándor Kolumbán, István Vajk, Johan Schoukens

Hypothesis testing methods that do not rely on exact distribution assumptions have been emerging lately. The method of sign-perturbed sums (SPS) is capable of characterizing confidence regions with exact confidence levels for linear regression and linear dynamical systems parameter estimation problems if the noise distribution is symmetric. This paper describes a general family of hypothesis testing methods that have an exact user chosen confidence level based on finite sample count and without relying on an assumed noise distribution. It is shown that the SPS method belongs to this family and we provide another hypothesis test for the case where the symmetry assumption is replaced with exchangeability. In the case of linear regression problems it is shown that the confidence regions are connected, bounded and possibly non-convex sets in both cases. To highlight the importance of understanding the structure of confidence regions corresponding to such hypothesis tests it is shown that confidence sets for linear dynamical systems parameter estimates generated using the SPS method can have non-connected parts, which have far reaching consequences.