PRApr 16
Quantitative Stability of Many-Marginal Schrodinger BridgeRentian Yao, Young-Heon Kim, Geoffrey Schiebinger
In this paper, we explore quantitative stability of multi-marginal Schrödinger bridges with respect to the marginal constraints. We focus on the case where the number of marginal constraints is large (i.e. ``many-marginals"). When this number increases, we show that the Kullback--Leibler (KL) divergence between two multi-marginal Schrödinger bridges, as measures on the path space, can be asymptotically bounded by the terminal marginal KL divergence and a time-integrated squared discrepancy {that combines} Wasserstein-2 geodesic velocity fields with a log-density gradient term. Our stability upper bound is also asymptotically tight: it converges to zero as the number of marginal constraints increases with unperturbed marginal constraints. To the best of our knowledge, this is the first such stability result that addresses the many-marginal regime, giving error estimates that are asymptotically independent of the number of marginals. To achieve our result, the key step is to derive an asymptotic expansion (of order $k\ge 2$) of Schrödinger potentials with respect to a diminishing regularization coefficient. This result can also be applied to deriving asymptotic expansions of entropic Brenier maps in entropic optimal self-transport problems. As byproducts of our analyses, we also establish the asymptotic expansion of entropic optimal transport cost with respect to the diminishing regularization coefficient when two marginal constraints are sufficiently close. We also prove a stability property of the Schrödinger functional.
CVMar 2
Safety-Guided Flow (SGF): A Unified Framework for Negative Guidance in Safe GenerationMingyu Kim, Young-Heon Kim, Mijung Park
Safety mechanisms for diffusion and flow models have recently been developed along two distinct paths. In robot planning, control barrier functions are employed to guide generative trajectories away from obstacles at every denoising step by explicitly imposing geometric constraints. In parallel, recent data-driven, negative guidance approaches have been shown to suppress harmful content and promote diversity in generated samples. However, they rely on heuristics without clearly stating when safety guidance is actually necessary. In this paper, we first introduce a unified probabilistic framework using a Maximum Mean Discrepancy (MMD) potential for image generation tasks that recasts both Shielded Diffusion and Safe Denoiser as instances of our energy-based negative guidance against unsafe data samples. Furthermore, we leverage control-barrier functions analysis to justify the existence of a critical time window in which negative guidance must be strong; outside of this window, the guidance should decay to zero to ensure safe and high-quality generation. We evaluate our unified framework on several realistic safe generation scenarios, confirming that negative guidance should be applied in the early stages of the denoising process for successful safe generation.
LGSep 25, 2024
Monge-Kantorovich Fitting With Sobolev BudgetsForest Kobayashi, Jonathan Hayase, Young-Heon Kim
Given $m < n$, we consider the problem of ``best'' approximating an $n\text{-d}$ probability measure $ρ$ via an $m\text{-d}$ measure $ν$ such that $\mathrm{supp}\ ν$ has bounded total ``complexity.'' When $ρ$ is concentrated near an $m\text{-d}$ set we may interpret this as a manifold learning problem with noisy data. However, we do not restrict our analysis to this case, as the more general formulation has broader applications. We quantify $ν$'s performance in approximating $ρ$ via the Monge-Kantorovich (also called Wasserstein) $p$-cost $\mathbb{W}_p^p(ρ, ν)$, and constrain the complexity by requiring $\mathrm{supp}\ ν$ to be coverable by an $f : \mathbb{R}^{m} \to \mathbb{R}^{n}$ whose $W^{k,q}$ Sobolev norm is bounded by $\ell \geq 0$. This allows us to reformulate the problem as minimizing a functional $\mathscr J_p(f)$ under the Sobolev ``budget'' $\ell$. This problem is closely related to (but distinct from) principal curves with length constraints when $m=1, k = 1$ and an unsupervised analogue of smoothing splines when $k > 1$. New challenges arise from the higher-order differentiability condition. We study the ``gradient'' of $\mathscr J_p$, which is given by a certain vector field that we call the barycenter field, and use it to prove a nontrivial (almost) strict monotonicity result. We also provide a natural discretization scheme and establish its consistency. We use this scheme as a toy model for a generative learning task, and by analogy, propose novel interpretations for the role regularization plays in improving training.
MLFeb 18, 2021
Towards a mathematical theory of trajectory inferenceHugo Lavenant, Stephen Zhang, Young-Heon Kim et al.
We devise a theoretical framework and a numerical method to infer trajectories of a stochastic process from samples of its temporal marginals. This problem arises in the analysis of single cell RNA-sequencing data, which provide high dimensional measurements of cell states but cannot track the trajectories of the cells over time. We prove that for a class of stochastic processes it is possible to recover the ground truth trajectories from limited samples of the temporal marginals at each time-point, and provide an efficient algorithm to do so in practice. The method we develop, Global Waddington-OT (gWOT), boils down to a smooth convex optimization problem posed globally over all time-points involving entropy-regularized optimal transport. We demonstrate that this problem can be solved efficiently in practice and yields good reconstructions, as we show on several synthetic and real datasets.