Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalities
This work provides theoretical guarantees for a recent optimization algorithm, addressing convergence issues in latent variable models, though it is incremental as it extends existing inequalities to a new setting.
The paper proves non-asymptotic error bounds for particle gradient descent (PGD), a method for maximum likelihood estimation in large latent variable models, by showing exponential convergence under generalized log-Sobolev and Polyak–Łojasiewicz inequalities, with results applicable to models with strongly concave log-likelihoods.
We prove non-asymptotic error bounds for particle gradient descent (PGD, Kuntz et al., 2023), a recently introduced algorithm for maximum likelihood estimation of large latent variable models obtained by discretizing a gradient flow of the free energy. We begin by showing that the flow converges exponentially fast to the free energy's minimizers for models satisfying a condition that generalizes both the log-Sobolev and the Polyak--Łojasiewicz inequalities (LSI and PŁI, respectively). We achieve this by extending a result well-known in the optimal transport literature (that the LSI implies the Talagrand inequality) and its counterpart in the optimization literature (that the PŁI implies the so-called quadratic growth condition), and applying the extension to our new setting. We also generalize the Bakry--Émery Theorem and show that the LSI/PŁI extension holds for models with strongly concave log-likelihoods. For such models, we further control PGD's discretization error and obtain the non-asymptotic error bounds. While we are motivated by the study of PGD, we believe that the inequalities and results we extend may be of independent interest.