Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
This addresses statistical inference challenges in high-dimensional nonlinear regression, which is incremental over existing sparse linear methods.
The paper tackles parameter estimation and inference in sparse nonlinear regression models where data follow y = f(x^⊤β*) + ε with nonlinear f, proposing an ℓ₁-regularized least-squares estimator. They prove that under mild conditions, every stationary point achieves optimal statistical convergence rates, and provide an efficient algorithm with valid hypothesis tests and confidence intervals for low-dimensional components.
We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y = f( x^\top β^* ) + ε$, where $f$ is nonlinear. To recover $β^*$, we propose an $\ell_1$-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of $f$. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. In addition, we provide an efficient algorithm that provably converges to a stationary point. We also access the uncertainty of the obtained estimator. Specifically, based on any stationary point of the objective, we construct valid hypothesis tests and confidence intervals for the low dimensional components of the high-dimensional parameter $β^*$. Detailed numerical results are provided to back up our theory.