Minimax Semiparametric Learning With Approximate Sparsity
This work addresses a gap in statistical theory for high-dimensional semi-parametric learning, providing foundational insights for researchers in statistics and machine learning dealing with approximate sparsity.
The paper tackles the problem of estimating linear functionals in high-dimensional contexts where models are only approximately sparse, rather than exactly sparse, and derives minimax rates for regression slope and average derivative, finding these bounds to be substantially larger than in low-dimensional settings, with the proposed estimator achieving minimax optimal rates.
Estimating linear, mean-square continuous functionals is a pivotal challenge in statistics. In high-dimensional contexts, this estimation is often performed under the assumption of exact model sparsity, meaning that only a small number of parameters are precisely non-zero. This excludes models where linear formulations only approximate the underlying data distribution, such as nonparametric regression methods that use basis expansion such as splines, kernel methods or polynomial regressions. Many recent methods for root-$n$ estimation have been proposed, but the implications of exact model sparsity remain largely unexplored. In particular, minimax optimality for models that are not exactly sparse has not yet been developed. This paper formalizes the concept of approximate sparsity through classical semi-parametric theory. We derive minimax rates under this formulation for a regression slope and an average derivative, finding these bounds to be substantially larger than those in low-dimensional, semi-parametric settings. We identify several new phenomena. We discover new regimes where rate double robustness does not hold, yet root-$n$ estimation is still possible. In these settings, we propose an estimator that achieves minimax optimal rates. Our findings further reveal distinct optimality boundaries for ordered versus unordered nonparametric regression estimation.