Fast and Interpretable Autoregressive Estimation with Neural Network Backpropagation
This provides a faster and more reliable alternative for time series analysts using AR models, though it is incremental as it applies an existing optimization method to a specific problem.
The paper tackled the computational expense and convergence issues in autoregressive (AR) model estimation by proposing a neural network formulation that embeds AR structure into a feedforward network, enabling coefficient estimation via backpropagation while preserving interpretability. Results showed the method consistently recovered coefficients for all 125,000 synthetic series, with a median speedup of 12.6x and up to 34.2x, while Conditional Maximum Likelihood failed to converge in about 55% of cases.
Autoregressive (AR) models remain widely used in time series analysis due to their interpretability, but convencional parameter estimation methods can be computationally expensive and prone to convergence issues. This paper proposes a Neural Network (NN) formulation of AR estimation by embedding the autoregressive structure directly into a feedforward NN, enabling coefficient estimation through backpropagation while preserving interpretability. Simulation experiments on 125,000 synthetic AR(p) time series with short-term dependence (1 <= p <= 5) show that the proposed NN-based method consistently recovers model coefficients for all series, while Conditional Maximum Likelihood (CML) fails to converge in approximately 55% of cases. When both methods converge, estimation accuracy is comparable with negligible differences in relative error, R2 and, perplexity/likelihood. However, when CML fails, the NN-based approach still provides reliable estimates. In all cases, the NN estimator achieves substantial computational gains, reaching a median speedup of 12.6x and up to 34.2x for higher model orders. Overall, results demonstrate that gradient-descent NN optimization can provide a fast and efficient alternative for interpretable AR parameter estimation.