Koen Tiels

Abhishek Chandra, Bram Daniels, Mitrofan Curti et al.

This article presents an approach for modelling hysteresis in piezoelectric materials, that leverages recent advancements in machine learning, particularly in sparse-regression techniques. While sparse regression has previously been used to model various scientific and engineering phenomena, its application to nonlinear hysteresis modelling in piezoelectric materials has yet to be explored. The study employs the least-squares algorithm with a sequential threshold to model the dynamic system responsible for hysteresis, resulting in a concise model that accurately predicts hysteresis for both simulated and experimental piezoelectric material data. Several numerical experiments are performed, including learning butterfly-shaped hysteresis and modelling real-world hysteresis data for a piezoelectric actuator. The presented approach is compared to traditional regression-based and neural network methods, demonstrating its efficiency and robustness. Source code is available at https://github.com/chandratue/SmartHysteresis

Abhishek Chandra, Bram Daniels, Mitrofan Curti et al.

Hysteresis modeling is crucial to comprehend the behavior of magnetic devices, facilitating optimal designs. Hitherto, deep learning-based methods employed to model hysteresis, face challenges in generalizing to novel input magnetic fields. This paper addresses the generalization challenge by proposing neural operators for modeling constitutive laws that exhibit magnetic hysteresis by learning a mapping between magnetic fields. In particular, three neural operators-deep operator network, Fourier neural operator, and wavelet neural operator-are employed to predict novel first-order reversal curves and minor loops, where novel means they are not used to train the model. In addition, a rate-independent Fourier neural operator is proposed to predict material responses at sampling rates different from those used during training to incorporate the rate-independent characteristics of magnetic hysteresis. The presented numerical experiments demonstrate that neural operators efficiently model magnetic hysteresis, outperforming the traditional neural recurrent methods on various metrics and generalizing to novel magnetic fields. The findings emphasize the advantages of using neural operators for modeling hysteresis under varying magnetic conditions, underscoring their importance in characterizing magnetic material based devices. The codes related to this paper are at github.com/chandratue/magnetic_hysteresis_neural_operator.

Koen Tiels, Johan Schoukens

Many nonlinear systems can be described by a Wiener-Schetzen model. In this model, the linear dynamics are formulated in terms of orthonormal basis functions (OBFs). The nonlinearity is modeled by a multivariate polynomial. In general, an infinite number of OBFs is needed for an exact representation of the system. This paper considers the approximation of a Wiener system with finite-order infinite impulse response dynamics and a polynomial nonlinearity. We propose to use a limited number of generalized OBFs (GOBFs). The pole locations, needed to construct the GOBFs, are estimated via the best linear approximation of the system. The coefficients of the multivariate polynomial are determined with a linear regression. This paper provides a convergence analysis for the proposed identification scheme. It is shown that the estimated output converges in probability to the exact output. Fast convergence rates, in the order $O_p({N_F}^{-n_{rep}/2})$, can be achieved, with $N_F$ the number of excited frequencies and $n_{rep}$ the number of repetitions of the GOBFs.

Johan Schoukens, Rik Pintelon, Yves Rolain et al.

In this paper we show that it is possible to retrieve structural information about complex block-oriented nonlinear systems, starting from linear approximations of the nonlinear system around different setpoints.The key idea is to monitor the movements of the poles and zeros of the linearized models and to reduce the number of candidate models on the basis of these observations. Besides the well known open loop single branch Wiener-, Hammerstein-, and Wiener-Hammerstein systems, we also cover a number of more general structures like parallel (multi branch) Wiener-Hammerstein models, and closed loop block oriented models, including linear fractional representation (LFR) models.

Jan Decuyper, Tim De Troyer, Mark Runacres et al.

The flow-induced vibration of bluff bodies is an important problem of many marine, civil, or mechanical engineers. In the design phase of such structures, it is vital to obtain good predictions of the fluid forces acting on the structure. Current methods rely on computational fluid dynamic simulations (CFD), with a too high computational cost to be effectively used in the design phase or for control applications. Alternative methods use heuristic mathematical models of the fluid forces, but these lack the accuracy (they often assume the system to be linear) or flexibility to be useful over a wide operating range. In this work we show that it is possible to build an accurate, flexible and low-computational-cost mathematical model using nonlinear system identification techniques. This model is data driven: it is trained over a user-defined region of interest using data obtained from experiments or simulations, or both. Here we use a Van der Pol oscillator as well as CFD simulations of an oscillating circular cylinder to generate the training data. Then a discrete-time polynomial nonlinear state-space model is fit to the data. This model relates the oscillation of the cylinder to the force that the fluid exerts on the cylinder. The model is finally validated over a wide range of oscillation frequencies and amplitudes, both inside and outside the so-called lock-in region. We show that forces simulated by the model are in good agreement with the data obtained from CFD.

Alireza Fakhrizadeh Esfahani, Philippe Dreesen, Koen Tiels et al.

Hysteresis is a highly nonlinear phenomenon, showing up in a wide variety of science and engineering problems. The identification of hysteretic systems from input-output data is a challenging task. Recent work on black-box polynomial nonlinear state-space modeling for hysteresis identification has provided promising results, but struggles with a large number of parameters due to the use of multivariate polynomials. This drawback is tackled in the current paper by applying a decoupling approach that results in a more parsimonious representation involving univariate polynomials. This work is carried out numerically on input-output data generated by a Bouc-Wen hysteretic model and follows up on earlier work of the authors. The current article discusses the polynomial decoupling approach and explores the selection of the number of univariate polynomials with the polynomial degree, as well as the connections with neural network modeling. We have found that the presented decoupling approach is able to reduce the number of parameters of the full nonlinear model up to about 50\%, while maintaining a comparable output error level.

Koen Tiels, Maarten Schoukens, Johan Schoukens

Block-oriented models are often used to model nonlinear systems. These models consist of linear dynamic (L) and nonlinear static (N) sub-blocks. This paper addresses the generation of initial estimates for a Wiener-Hammerstein model (LNL cascade). While it is easy to measure the product of the two linear blocks using a Gaussian excitation and linear identification methods, it is difficult to split the global dynamics over the individual blocks. This paper first proposes a well-designed multisine excitation with pairwise coupled random phases. Next, a modified best linear approximation is estimated on a shifted frequency grid. It is shown that this procedure creates a shift of the input dynamics with a known frequency offset, while the output dynamics do not shift. The resulting transfer function, which has complex coefficients due to the frequency shift, is estimated with a modified frequency domain estimation method. The identified poles and zeros can be assigned to either the input or output dynamics. Once this is done, it is shown in the literature that the remaining initialization problem can be solved much easier than the original one. The method is illustrated on experimental data obtained from the Wiener-Hammerstein benchmark system.

Maarten Schoukens, Koen Tiels

Block-oriented nonlinear models are popular in nonlinear system identification because of their advantages of being simple to understand and easy to use. Many different identification approaches were developed over the years to estimate the parameters of a wide range of block-oriented nonlinear models. One class of these approaches uses linear approximations to initialize the identification algorithm. The best linear approximation framework and the $ε$-approximation framework, or equivalent frameworks, allow the user to extract important information about the system, guide the user in selecting good candidate model structures and orders, and prove to be a good starting point for nonlinear system identification algorithms. This paper gives an overview of the different block-oriented nonlinear models that can be identified using linear approximations, and of the identification algorithms that have been developed in the past. A non-exhaustive overview of the most important other block-oriented nonlinear system identification approaches is also provided throughout this paper.

Anna Marconato, Maarten Schoukens, Koen Tiels et al.

In this paper, several advanced data-driven nonlinear identification techniques are compared on a specific problem: a simplified glucoregulatory system modeling example. This problem represents a challenge in the development of an artificial pancreas for T1DM treatment, since for this application good nonlinear models are needed to design accurate closed-loop controllers to regulate the glucose level in the blood. Block-oriented as well as state-space models are used to describe both the dynamics and the nonlinear behavior of the insulin-glucose system, and the advantages and drawbacks of each method are pointed out. The obtained nonlinear models are accurate in simulating the patient's behavior, and some of them are also sufficiently simple to be considered in the implementation of a model-based controller to develop the artificial pancreas.

Rishi Relan, Koen Tiels, Jean-Marc Timmermans et al.

Battery short-term electrical impedance behavior varies between linear, linear time-varying, or nonlinear at different operating conditions. Data-based electrical impedance modeling techniques often model the battery as a linear time-invariant system at all operating conditions. In addition, these techniques require extensive and time consuming experimentation. Often due to sensor failures during experiments, constraints in data acquisition hardware, varying operating conditions, and the slow dynamics of the battery, it is not always possible to acquire data in a single experiment. Hence, multiple experiments must be performed. In this letter, a local polynomial approach is proposed to estimate nonparametrically the best linear approximation of the electrical impedance affected by varying levels of nonlinear distortion, from a series of input current and output voltage data subrecords of arbitrary length.

Rishi Relan, Koen Tiels, Anna Marconato et al.

Many real world systems exhibit a quasi linear or weakly nonlinear behavior during normal operation, and a hard saturation effect for high peaks of the input signal. In this paper, a methodology to identify a parsimonious discrete-time nonlinear state space model (NLSS) for the nonlinear dynamical system with relatively short data record is proposed. The capability of the NLSS model structure is demonstrated by introducing two different initialisation schemes, one of them using multivariate polynomials. In addition, a method using first-order information of the multivariate polynomials and tensor decomposition is employed to obtain the parsimonious decoupled representation of the set of multivariate real polynomials estimated during the identification of NLSS model. Finally, the experimental verification of the model structure is done on the cascaded water-benchmark identification problem.

Abhishek Chandra, Taniya Kapoor, Bram Daniels et al.

Hysteresis is a ubiquitous phenomenon in science and engineering; its modeling and identification are crucial for understanding and optimizing the behavior of various systems. We develop an ordinary differential equation-based recurrent neural network (RNN) approach to model and quantify the hysteresis, which manifests itself in sequentiality and history-dependence. Our neural oscillator, HystRNN, draws inspiration from coupled-oscillatory RNN and phenomenological hysteresis models to update the hidden states. The performance of HystRNN is evaluated to predict generalized scenarios, involving first-order reversal curves and minor loops. The findings show the ability of HystRNN to generalize its behavior to previously untrained regions, an essential feature that hysteresis models must have. This research highlights the advantage of neural oscillators over the traditional RNN-based methods in capturing complex hysteresis patterns in magnetic materials, where traditional rate-dependent methods are inadequate to capture intrinsic nonlinearity.

Dominik Baumann, Krzysztof Kowalczyk, Koen Tiels et al.

Safety is an essential asset when learning control policies for physical systems, as violating safety constraints during training can lead to expensive hardware damage. In response to this need, the field of safe learning has emerged with algorithms that can provide probabilistic safety guarantees without knowledge of the underlying system dynamics. Those algorithms often rely on Gaussian process inference. Unfortunately, Gaussian process inference scales cubically with the number of data points, limiting applicability to high-dimensional and embedded systems. In this paper, we propose a safe learning algorithm that provides probabilistic safety guarantees but leverages the Nadaraya-Watson estimator instead of Gaussian processes. For the Nadaraya-Watson estimator, we can reach logarithmic scaling with the number of data points. We provide theoretical guarantees for the estimates, embed them into a safe learning algorithm, and show numerical experiments on a simulated seven-degrees-of-freedom robot manipulator.

Dominik Baumann, Krzysztof Kowalczyk, Cristian R. Rojas et al.

Applying machine learning methods to physical systems that are supposed to act in the real world requires providing safety guarantees. However, methods that include such guarantees often come at a high computational cost, making them inapplicable to large datasets and embedded devices with low computational power. In this paper, we propose CoLSafe, a computationally lightweight safe learning algorithm whose computational complexity grows sublinearly with the number of data points. We derive both safety and optimality guarantees and showcase the effectiveness of our algorithm on a seven-degrees-of-freedom robot arm.

Abhishek Chandra, Taniya Kapoor, Mitrofan Curti et al.

Complex piezoelectric systems are foundational in industrial applications. Their performance, however, is challenged by the nonlinear voltage-displacement hysteretic relationships. Efficient characterization methods are, therefore, essential for reliable design, monitoring, and maintenance. Recently proposed neural operator methods serve as surrogates for system characterization but face two pressing issues: interpretability and generalizability. State-of-the-art (SOTA) neural operators are black-boxes, providing little insight into the learned operator. Additionally, generalizing them to novel voltages and predicting displacement profiles beyond the training domain is challenging, limiting their practical use. To address these limitations, this paper proposes a neuro-symbolic operator (NSO) framework that derives the analytical operators governing hysteretic relationships. NSO first learns a Fourier neural operator mapping voltage fields to displacement profiles, followed by a library-based sparse model discovery method, generating white-box parsimonious models governing the underlying hysteresis. These models enable accurate and interpretable prediction of displacement profiles across varying and out-of-distribution voltage fields, facilitating generalizability. The potential of NSO is demonstrated by accurately predicting voltage-displacement hysteresis, including butterfly-shaped relationships. Moreover, NSO predicts displacement profiles even for noisy and low-fidelity voltage data, emphasizing its robustness. The results highlight the advantages of NSO compared to SOTA neural operators and model discovery methods on several evaluation metrics. Consequently, NSO contributes to characterizing complex piezoelectric systems while improving the interpretability and generalizability of neural operators, essential for design, monitoring, maintenance, and other real-world scenarios.

Carl Andersson, Antônio H. Ribeiro, Koen Tiels et al.

Recent developments within deep learning are relevant for nonlinear system identification problems. In this paper, we establish connections between the deep learning and the system identification communities. It has recently been shown that convolutional architectures are at least as capable as recurrent architectures when it comes to sequence modeling tasks. Inspired by these results we explore the explicit relationships between the recently proposed temporal convolutional network (TCN) and two classic system identification model structures; Volterra series and block-oriented models. We end the paper with an experimental study where we provide results on two real-world problems, the well-known Silverbox dataset and a newer dataset originating from ground vibration experiments on an F-16 fighter aircraft.

Antônio H. Ribeiro, Koen Tiels, Luis A. Aguirre et al.

The exploding and vanishing gradient problem has been the major conceptual principle behind most architecture and training improvements in recurrent neural networks (RNNs) during the last decade. In this paper, we argue that this principle, while powerful, might need some refinement to explain recent developments. We refine the concept of exploding gradients by reformulating the problem in terms of the cost function smoothness, which gives insight into higher-order derivatives and the existence of regions with many close local minima. We also clarify the distinction between vanishing gradients and the need for the RNN to learn attractors to fully use its expressive power. Through the lens of these refinements, we shed new light on recent developments in the RNN field, namely stable RNN and unitary (or orthogonal) RNNs.

Antônio H. Ribeiro, Koen Tiels, Jack Umenberger et al.

We shed new light on the \textit{smoothness} of optimization problems arising in prediction error parameter estimation of linear and nonlinear systems. We show that for regions of the parameter space where the model is not contractive, the Lipschitz constant and $β$-smoothness of the objective function might blow up exponentially with the simulation length, making it hard to numerically find minima within those regions or, even, to escape from them. In addition to providing theoretical understanding of this problem, this paper also proposes the use of multiple shooting as a viable solution. The proposed method minimizes the error between a prediction model and the observed values. Rather than running the prediction model over the entire dataset, multiple shooting splits the data into smaller subsets and runs the prediction model over each subset, making the simulation length a design parameter and making it possible to solve problems that would be infeasible using a standard approach. The equivalence to the original problem is obtained by including constraints in the optimization. The new method is illustrated by estimating the parameters of nonlinear systems with chaotic or unstable behavior, as well as neural networks. We also present a comparative analysis of the proposed method with multi-step-ahead prediction error minimization.