Estimation of Non-Crossing Quantile Regression Process with Deep ReQU Neural Networks
This work addresses the issue of quantile crossing in regression for statistical modeling, offering an incremental improvement with a novel penalty and approximation bounds.
The authors tackled the problem of estimating non-crossing quantile regression processes in nonseparable models by proposing a penalized nonparametric approach using deep ReQU neural networks, achieving competitive or superior performance compared to existing methods like reproducing kernels and random forests in numerical experiments.
We propose a penalized nonparametric approach to estimating the quantile regression process (QRP) in a nonseparable model using rectifier quadratic unit (ReQU) activated deep neural networks and introduce a novel penalty function to enforce non-crossing of quantile regression curves. We establish the non-asymptotic excess risk bounds for the estimated QRP and derive the mean integrated squared error for the estimated QRP under mild smoothness and regularity conditions. To establish these non-asymptotic risk and estimation error bounds, we also develop a new error bound for approximating $C^s$ smooth functions with $s >0$ and their derivatives using ReQU activated neural networks. This is a new approximation result for ReQU networks and is of independent interest and may be useful in other problems. Our numerical experiments demonstrate that the proposed method is competitive with or outperforms two existing methods, including methods using reproducing kernels and random forests, for nonparametric quantile regression.