Minimum Stein Discrepancy Estimators
This provides a flexible framework for designing estimators with specific properties, addressing incremental improvements in statistical estimation for models with difficult densities.
The authors tackled the problem of parameter estimation when maximum likelihood is infeasible by unifying existing techniques as minimum Stein discrepancy estimators, leading to new estimators (DKSD and DSM) with proven consistency, asymptotic normality, and robustness, and applied them to challenging densities like non-smooth or heavy-tailed ones.
When maximum likelihood estimation is infeasible, one often turns to score matching, contrastive divergence, or minimum probability flow to obtain tractable parameter estimates. We provide a unifying perspective of these techniques as minimum Stein discrepancy estimators, and use this lens to design new diffusion kernel Stein discrepancy (DKSD) and diffusion score matching (DSM) estimators with complementary strengths. We establish the consistency, asymptotic normality, and robustness of DKSD and DSM estimators, then derive stochastic Riemannian gradient descent algorithms for their efficient optimisation. The main strength of our methodology is its flexibility, which allows us to design estimators with desirable properties for specific models at hand by carefully selecting a Stein discrepancy. We illustrate this advantage for several challenging problems for score matching, such as non-smooth, heavy-tailed or light-tailed densities.