Regularising NARX models with multi-task learning
This addresses the problem of poor generalization in time-series prediction for domains using NARX models, though it appears incremental as it builds on existing regularization techniques.
The paper tackles overfitting in Nonlinear Auto-Regressive with eXogenous inputs (NARX) models by proposing a multi-task learning approach that predicts outputs at current and future lead times, resulting in a lower Normalised Mean Square Error (NMSE) compared to independent learners, especially under high noise conditions.
A Nonlinear Auto-Regressive with eXogenous inputs (NARX) model can be used to describe time-varying processes; where the output depends on both previous outputs and current/previous external input variables. One limitation of NARX models is their propensity to overfit and result in poor generalisation for future predictions. The proposed method to help to overcome the issue of overfitting is a NARX model which predicts outputs at both the current time and several lead times into the future. This is a form of multi-task learner (MTL); whereby the lead time outputs will regularise the current time output. This work shows that for high noise level, MTL can be used to regularise NARX with a lower Normalised Mean Square Error (NMSE) compared to the NMSE of the independent learner counterpart.