Interpretation and Generalization of Score Matching
This work addresses theoretical gaps in score matching, offering incremental improvements for researchers in density estimation and machine learning.
The paper formalizes the connection between maximum likelihood and score matching, showing that score matching yields more robust parameters with noisy data, and generalizes score matching to discrete data models.
Score matching is a recently developed parameter learning method that is particularly effective to complicated high dimensional density models with intractable partition functions. In this paper, we study two issues that have not been completely resolved for score matching. First, we provide a formal link between maximum likelihood and score matching. Our analysis shows that score matching finds model parameters that are more robust with noisy training data. Second, we develop a generalization of score matching. Based on this generalization, we further demonstrate an extension of score matching to models of discrete data.