Constrained Exploration via Reflected Replica Exchange Stochastic Gradient Langevin Dynamics
This work addresses efficiency problems in sampling for non-convex learning, particularly in large-scale datasets, but is incremental as it builds on existing reSGLD methods.
The paper tackled the stagnation issue in replica exchange stochastic gradient Langevin dynamics (reSGLD) when exploring non-convex distributions by proposing reflected reSGLD (r2SGLD), which uses reflection steps within a bounded domain to improve mixing rates, showing a quadratic enhancement in theory and validated through experiments on dynamical systems, multi-modal distributions, and image classification.
Replica exchange stochastic gradient Langevin dynamics (reSGLD) is an effective sampler for non-convex learning in large-scale datasets. However, the simulation may encounter stagnation issues when the high-temperature chain delves too deeply into the distribution tails. To tackle this issue, we propose reflected reSGLD (r2SGLD): an algorithm tailored for constrained non-convex exploration by utilizing reflection steps within a bounded domain. Theoretically, we observe that reducing the diameter of the domain enhances mixing rates, exhibiting a $\textit{quadratic}$ behavior. Empirically, we test its performance through extensive experiments, including identifying dynamical systems with physical constraints, simulations of constrained multi-modal distributions, and image classification tasks. The theoretical and empirical findings highlight the crucial role of constrained exploration in improving the simulation efficiency.