Robust Score Matching
This work addresses the problem of robust parameter estimation for researchers and practitioners in statistics and machine learning, particularly in non-Gaussian exponential family graphical models, though it is incremental as it builds on existing score matching techniques.
The authors tackled the problem of parameter estimation in contaminated data settings by developing a robust score matching method using the geometric median of means, which yields consistent estimates and retains convexity in exponential family models. They demonstrated that this method performs comparably to standard score matching without contamination but greatly outperforms it with contamination, as shown in numerical experiments and a precipitation dataset.
Proposed in Hyvärinen (2005), score matching is a parameter estimation procedure that does not require computation of distributional normalizing constants. In this work we utilize the geometric median of means to develop a robust score matching procedure that yields consistent parameter estimates in settings where the observed data has been contaminated. A special appeal of the proposed method is that it retains convexity in exponential family models. The new method is therefore particularly attractive for non-Gaussian, exponential family graphical models where evaluation of normalizing constants is intractable. Support recovery guarantees for such models when contamination is present are provided. Additionally, support recovery is studied in numerical experiments and on a precipitation dataset. We demonstrate that the proposed robust score matching estimator performs comparably to the standard score matching estimator when no contamination is present but greatly outperforms this estimator in a setting with contamination.