Optimality of Maximum Likelihood for Log-Concave Density Estimation and Bounded Convex Regression
This work addresses theoretical limits in nonparametric statistics for high-dimensional data, providing foundational insights into optimal estimation rates, but it is incremental as it extends known results to higher dimensions.
The paper tackles the problem of estimating log-concave densities and bounded convex regression, showing that maximum likelihood estimators achieve optimal risk rates of Θ_d(n^{-2/(d+1)}) for dimensions d ≥ 4, extending previous results known only for d ≤ 3, and proves a sharp minimax bound of Θ_d(n^{-2/(d+4)}) for total variation distance.
In this paper, we study two problems: (1) estimation of a $d$-dimensional log-concave distribution and (2) bounded multivariate convex regression with random design with an underlying log-concave density or a compactly supported distribution with a continuous density. First, we show that for all $d \ge 4$ the maximum likelihood estimators of both problems achieve an optimal risk of $Θ_d(n^{-2/(d+1)})$ (up to a logarithmic factor) in terms of squared Hellinger distance and $L_2$ squared distance, respectively. Previously, the optimality of both these estimators was known only for $d\le 3$. We also prove that the $ε$-entropy numbers of the two aforementioned families are equal up to logarithmic factors. We complement these results by proving a sharp bound $Θ_d(n^{-2/(d+4)})$ on the minimax rate (up to logarithmic factors) with respect to the total variation distance. Finally, we prove that estimating a log-concave density - even a uniform distribution on a convex set - up to a fixed accuracy requires the number of samples \emph{at least} exponential in the dimension. We do that by improving the dimensional constant in the best known lower bound for the minimax rate from $2^{-d}\cdot n^{-2/(d+1)}$ to $c\cdot n^{-2/(d+1)}$ (when $d\geq 2$).