Nonparametric Estimation of Renyi Divergence and Friends
This work addresses the estimation of divergence measures for statistical inference, which is incremental as it builds on existing nonparametric methods with specific theoretical improvements.
The paper tackles the problem of nonparametric estimation of L2, Renyi-α, and Tsallis-α divergences between continuous distributions by constructing estimators for integral functionals of densities, achieving a parametric convergence rate of n^{-1/2} when densities have smoothness at least d/4, with minimax lower bounds confirming this condition.
We consider nonparametric estimation of $L_2$, Renyi-$α$ and Tsallis-$α$ divergences between continuous distributions. Our approach is to construct estimators for particular integral functionals of two densities and translate them into divergence estimators. For the integral functionals, our estimators are based on corrections of a preliminary plug-in estimator. We show that these estimators achieve the parametric convergence rate of $n^{-1/2}$ when the densities' smoothness, $s$, are both at least $d/4$ where $d$ is the dimension. We also derive minimax lower bounds for this problem which confirm that $s > d/4$ is necessary to achieve the $n^{-1/2}$ rate of convergence. We validate our theoretical guarantees with a number of simulations.