Contraction and entropy production in continuous-time Sinkhorn dynamics
This work provides theoretical insights into Sinkhorn dynamics, which is incremental for researchers in optimal transport and machine learning.
The paper tackled the problem of analyzing the continuous-time limit of the Sinkhorn algorithm, showing that it induces a reversible Markov dynamics with a positive spectral gap and providing an exact identity for entropy production, which was previously only known to be nonpositive. It demonstrated practical applications, such as using a logarithmic Sobolev inequality for designing generative model latent spaces and as a stopping heuristic.
Recently, the vanishing-step-size limit of the Sinkhorn algorithm at finite regularization parameter $\varepsilon$ was shown to be a mirror descent in the space of probability measures. We give $L^2$ contraction criteria in two time-dependent metrics induced by the mirror Hessian, which reduce to the coercivity of certain conditional expectation operators. We then give an exact identity for the entropy production rate of the Sinkhorn flow, which was previously known only to be nonpositive. Examining this rate shows that the standard semigroup analysis of diffusion processes extends systematically to the Sinkhorn flow. We show that the flow induces a reversible Markov dynamics on the target marginal as an Onsager gradient flow. We define the Dirichlet form associated to its (nonlocal) infinitesimal generator, prove a Poincaré inequality for it, and show that the spectral gap is strictly positive along the Sinkhorn flow whenever $\varepsilon > 0$. Lastly, we show that the entropy decay is exponential if and only if a logarithmic Sobolev inequality (LSI) holds. We give for illustration two immediate practical use-cases for the Sinkhorn LSI: as a design principle for the latent space in which generative models are trained, and as a stopping heuristic for discrete-time algorithms.