Optimal Nonparametric Inference via Deep Neural Network
This addresses a theoretical bottleneck in statistical inference for researchers, providing optimal conditions for network architectures, though it is incremental as it builds on existing nonparametric estimation literature.
The paper tackles the suboptimal logarithmic factors in deep neural network performance for nonparametric estimation, showing these factors are unnecessary and deriving optimal upper bounds for the L^2 minimax risk without such sacrifices.
Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not necessary. We derive upper bounds for the $L^2$ minimax risk in nonparametric estimation. Sufficient conditions on network architectures are provided such that the upper bounds become optimal (without log-sacrifice). Our proof relies on an explicitly constructed network estimator based on tensor product B-splines. We also derive asymptotic distributions for the constructed network and a relating hypothesis testing procedure. The testing procedure is further proven as minimax optimal under suitable network architectures.