Deep regression learning from dependent observations with minimum error entropy principle
This work addresses regression problems with dependent data, which is common in time series or spatial applications, but it is incremental as it builds on existing minimax optimality results.
The paper tackles nonparametric regression from dependent (strongly mixing) observations using deep neural networks with the minimum error entropy principle, establishing that both proposed estimators achieve minimax optimal convergence rates (up to a logarithmic factor) for Gaussian error models.
This paper considers nonparametric regression from strongly mixing observations. The proposed approach is based on deep neural networks with minimum error entropy (MEE) principle. We study two estimators: the non-penalized deep neural network (NPDNN) and the sparse-penalized deep neural network (SPDNN) predictors. Upper bounds of the expected excess risk are established for both estimators over the classes of Hölder and composition Hölder functions. For the models with Gaussian error, the rates of the upper bound obtained match (up to a logarithmic factor) with the lower bounds established in \cite{schmidt2020nonparametric}, showing that both the MEE-based NPDNN and SPDNN estimators from strongly mixing data can achieve the minimax optimal convergence rate.