Probabilistic Gaussian Homotopy: A Probability-Space Continuation Framework for Nonconvex Optimization
This addresses optimization challenges in machine learning and related fields, offering a novel approach with practical improvements, though it appears incremental as it builds on existing homotopy and smoothing techniques.
The paper tackles nonconvex optimization by introducing Probabilistic Gaussian Homotopy (PGH), a framework that deforms Boltzmann distributions to bias gradients toward low-energy regions, and demonstrates strong performance on high-dimensional benchmarks and sparse recovery problems where traditional methods often fail.
We introduce Probabilistic Gaussian Homotopy (PGH), a probability-space continuation framework for nonconvex optimization. Unlike classical Gaussian homotopy, which smooths the objective and uniformly averages gradients, PGH deforms the associated Boltzmann distribution and induces Boltzmann-weighted aggregation of perturbed gradients, which exponentially biases descent directions toward low-energy regions. We show that PGH corresponds to a log-sum-exp (soft-min) homotopy that smooths a nonconvex objective at scale $λ>0$ and recovers the original objective as $λ\to 0$, yielding a posterior-mean generalization of the Moreau envelope, and we derive a dynamical system governing minimizer evolution along an annealed homotopy path. This establishes a principled connection between Gaussian continuation, Bayesian denoising, and diffusion-style smoothing. We further propose Probabilistic Gaussian Homotopy Optimization (PGHO), a practical stochastic algorithm based on Monte Carlo gradient estimation, and demonstrate strong performance on high-dimensional nonconvex benchmarks and sparse recovery problems where classical gradient methods and objective-space smoothing frequently fail.