Minimizing $f$-Divergences by Interpolating Velocity Fields
This work addresses overfitting issues in distribution approximation for machine learning tasks, offering a more reliable method for domain adaptation and missing data imputation.
The paper tackles the problem of approximating a target distribution by minimizing f-divergences, using interpolation to directly estimate velocity fields for Wasserstein Gradient Flow, which improves accuracy and is validated in domain adaptation and missing data imputation applications.
Many machine learning problems can be seen as approximating a \textit{target} distribution using a \textit{particle} distribution by minimizing their statistical discrepancy. Wasserstein Gradient Flow can move particles along a path that minimizes the $f$-divergence between the target and particle distributions. To move particles, we need to calculate the corresponding velocity fields derived from a density ratio function between these two distributions. Previous works estimated such density ratio functions and then differentiated the estimated ratios. These approaches may suffer from overfitting, leading to a less accurate estimate of the velocity fields. Inspired by non-parametric curve fitting, we directly estimate these velocity fields using interpolation techniques. We prove that our estimators are consistent under mild conditions. We validate their effectiveness using novel applications on domain adaptation and missing data imputation.