A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation
This work offers a theoretical framework for researchers in machine learning and statistics, but it is incremental as it unifies existing methods rather than introducing a new paradigm.
The paper tackles the problem of learning unnormalized distributions by providing a unified perspective on various methods through noise-contrastive estimation, establishing finite-sample convergence rates for exponential families under new regularity assumptions.
This paper studies a family of estimators based on noise-contrastive estimation (NCE) for learning unnormalized distributions. The main contribution of this work is to provide a unified perspective on various methods for learning unnormalized distributions, which have been independently proposed and studied in separate research communities, through the lens of NCE. This unified view offers new insights into existing estimators. Specifically, for exponential families, we establish the finite-sample convergence rates of the proposed estimators under a set of regularity assumptions, most of which are new.