Parallel Simulation for Log-concave Sampling and Score-based Diffusion Models
This addresses computational efficiency for large datasets in ML/statistics, though it appears incremental as it builds on existing parallel techniques.
The paper tackles the problem of high-dimensional sampling in machine learning by proposing a novel parallel method that reduces adaptive complexity from ̃O(log^2 d) to ̃O(log d), achieving optimal rates for log-concave sampling.
Sampling from high-dimensional probability distributions is fundamental in machine learning and statistics. As datasets grow larger, computational efficiency becomes increasingly important, particularly in reducing adaptive complexity, namely the number of sequential rounds required for sampling algorithms. While recent works have introduced several parallelizable techniques, they often exhibit suboptimal convergence rates and remain significantly weaker than the latest lower bounds for log-concave sampling. To address this, we propose a novel parallel sampling method that improves adaptive complexity dependence on dimension $d$ reducing it from $\widetilde{\mathcal{O}}(\log^2 d)$ to $\widetilde{\mathcal{O}}(\log d)$. which is even optimal for log-concave sampling with some specific adaptive complexity. Our approach builds on parallel simulation techniques from scientific computing.