Non-Autoregressive vs Autoregressive Neural Networks for System Identification
This addresses efficiency and accuracy issues in system identification for engineering and AI applications, but is incremental as it compares existing architectures.
The paper tackled the problem of using autoregressive methods in neural networks for system identification, showing that non-autoregressive implementations of GRU and TCN are significantly faster and at least as accurate, with the non-autoregressive GRU being the best neural network-based method and top black-box method in benchmarks without extrapolation.
The application of neural networks to non-linear dynamic system identification tasks has a long history, which consists mostly of autoregressive approaches. Autoregression, the usage of the model outputs of previous time steps, is a method of transferring a system state between time steps, which is not necessary for modeling dynamic systems with modern neural network structures, such as gated recurrent units (GRUs) and Temporal Convolutional Networks (TCNs). We compare the accuracy and execution performance of autoregressive and non-autoregressive implementations of a GRU and TCN on the simulation task of three publicly available system identification benchmarks. Our results show, that the non-autoregressive neural networks are significantly faster and at least as accurate as their autoregressive counterparts. Comparisons with other state-of-the-art black-box system identification methods show, that our implementation of the non-autoregressive GRU is the best performing neural network-based system identification method, and in the benchmarks without extrapolation, the best performing black-box method.