Optimal rates for density and mode estimation with expand-and-sparsify representations
This work addresses fundamental statistical problems for theoretical modeling, but it appears incremental as it applies an existing representation method to new tasks.
The paper tackles density and mode estimation using expand-and-sparsify representations, showing that a linear function of this representation yields a density estimator with minimax-optimal ℓ∞ convergence rates and algorithms for mode estimation achieve optimal rates up to logarithmic factors.
Expand-and-sparsify representations are a class of theoretical models that capture sparse representation phenomena observed in the sensory systems of many animals. At a high level, these representations map an input $x \in \mathbb{R}^d$ to a much higher dimension $m \gg d$ via random linear projections before zeroing out all but the $k \ll m$ largest entries. The result is a $k$-sparse vector in $\{0,1\}^m$. We study the suitability of this representation for two fundamental statistical problems: density estimation and mode estimation. For density estimation, we show that a simple linear function of the expand-and-sparsify representation produces an estimator with minimax-optimal $\ell_{\infty}$ convergence rates. In mode estimation, we provide simple algorithms on top of our density estimator that recover single or multiple modes at optimal rates up to logarithmic factors under mild conditions.