Minimax Optimal Rates of Estimation in High Dimensional Additive Models: Universal Phase Transition
This work addresses the fundamental problem of optimal estimation rates in high-dimensional additive models for statisticians and machine learning researchers, providing theoretical insights into phase transitions but is incremental as it builds on existing minimax theory.
The paper establishes minimax optimal convergence rates for estimation in high-dimensional additive models, revealing a universal phase transition: in the sparse regime, rates match those of high-dimensional linear regression, while in the smooth regime, they align with univariate function estimation, avoiding the curse of dimensionality.
We establish minimax optimal rates of convergence for estimation in a high dimensional additive model assuming that it is approximately sparse. Our results reveal an interesting phase transition behavior universal to this class of high dimensional problems. In the {\it sparse regime} when the components are sufficiently smooth or the dimensionality is sufficiently large, the optimal rates are identical to those for high dimensional linear regression, and therefore there is no additional cost to entertain a nonparametric model. Otherwise, in the so-called {\it smooth regime}, the rates coincide with the optimal rates for estimating a univariate function, and therefore they are immune to the "curse of dimensionality".