Generalization error bound for denoising score matching under relaxed manifold assumption
This provides theoretical foundations for denoising score matching in machine learning, addressing a key limitation in existing assumptions, but it is incremental as it builds on prior work with a relaxed assumption.
The paper tackles the problem of deriving theoretical guarantees for denoising score matching by relaxing the standard manifold assumption to allow samples to deviate from the manifold, resulting in non-asymptotic bounds on approximation and generalization errors with convergence rates determined by intrinsic dimension and validity even with polynomially growing ambient dimension.
We examine theoretical properties of the denoising score matching estimate. We model the density of observations with a nonparametric Gaussian mixture. We significantly relax the standard manifold assumption allowing the samples step away from the manifold. At the same time, we are still able to leverage a nice distribution structure. We derive non-asymptotic bounds on the approximation and generalization errors of the denoising score matching estimate. The rates of convergence are determined by the intrinsic dimension. Furthermore, our bounds remain valid even if we allow the ambient dimension grow polynomially with the sample size.