Statistical Properties of Deep Neural Networks with Dependent Data
It addresses theoretical guarantees for deep learning with dependent data, which is incremental but important for time-series applications.
This paper establishes statistical properties of deep neural network estimators under dependent data, providing convergence rates under nonstationary data and non-asymptotic error bounds under stationary β-mixing data for regression and classification tasks.
This paper establishes statistical properties of deep neural network (DNN) estimators under dependent data. Two general results for nonparametric sieve estimators directly applicable to DNN estimators are given. The first establishes rates for convergence in probability under nonstationary data. The second provides non-asymptotic probability bounds on $\mathcal{L}^{2}$-errors under stationary $β$-mixing data. I apply these results to DNN estimators in both regression and classification contexts imposing only a standard Hölder smoothness assumption. The DNN architectures considered are common in applications, featuring fully connected feedforward networks with any continuous piecewise linear activation function, unbounded weights, and a width and depth that grows with sample size. The framework provided also offers potential for research into other DNN architectures and time-series applications.