A hierarchical entropy method for the delocalization of bias in high-dimensional Langevin Monte Carlo
This provides incremental improvements in understanding bias delocalization for high-dimensional Monte Carlo methods, relevant for researchers in computational statistics and machine learning.
The paper tackles the bias in the unadjusted Langevin algorithm for high-dimensional sampling, showing that for distributions with sparse or weak interactions, the bias between low-dimensional marginals scales only with the lower dimension, not the full dimension, and they remove a logarithmic factor and relax assumptions compared to prior work.
The unadjusted Langevin algorithm is widely used for sampling from complex high-dimensional distributions. It is well known to be biased, with the bias typically scaling linearly with the dimension when measured in squared Wasserstein distance. However, the recent paper of Chen et al. (2024) identifies an intriguing new delocalization effect: For a class of distributions with sparse interactions, the bias between low-dimensional marginals scales only with the lower dimension, not the full dimension. In this work, we strengthen the results of Chen et al. (2024) in the sparse interaction regime by removing a logarithmic factor, measuring distance in relative entropy (a.k.a. KL-divergence), and relaxing the strong log-concavity assumption. In addition, we expand the scope of the delocalization phenomenon by showing that it holds for a class of distributions with weak interactions. Our proofs are based on a hierarchical analysis of the marginal relative entropies, inspired by the authors' recent work on propagation of chaos.