Adaptive learning of density ratios in RKHS
This work addresses a central problem in machine learning and statistics, offering an adaptive solution that improves estimation accuracy without prior knowledge of regularity, though it is incremental as it builds on existing density ratio estimation frameworks.
The paper tackles the problem of estimating the ratio of two probability densities from finite observations, a key task in machine learning with applications like two-sample testing and generative modeling, by analyzing methods that minimize a regularized Bregman divergence in an RKHS and proposing an adaptive parameter choice that achieves minimax optimal error rates in certain cases.
Estimating the ratio of two probability densities from finitely many observations of the densities is a central problem in machine learning and statistics with applications in two-sample testing, divergence estimation, generative modeling, covariate shift adaptation, conditional density estimation, and novelty detection. In this work, we analyze a large class of density ratio estimation methods that minimize a regularized Bregman divergence between the true density ratio and a model in a reproducing kernel Hilbert space (RKHS). We derive new finite-sample error bounds, and we propose a Lepskii type parameter choice principle that minimizes the bounds without knowledge of the regularity of the density ratio. In the special case of quadratic loss, our method adaptively achieves a minimax optimal error rate. A numerical illustration is provided.