Multi-marginal temporal Schrödinger Bridge Matching for video generation from unpaired data
This work addresses the challenge of inferring hidden dynamics from static data in scientific domains like biology, offering a scalable and principled method for video generation and trajectory inference.
The paper tackles the problem of reconstructing temporal evolution from static snapshots, such as in cellular differentiation or disease progression, by proposing Multi-Marginal temporal Schrödinger Bridge Matching (MMtSBM) for video generation from unpaired data, achieving state-of-the-art performance on real-world datasets like transcriptomic trajectory inference in 100 dimensions and recovering couplings in high-dimensional image settings.
Many natural dynamic processes -- such as in vivo cellular differentiation or disease progression -- can only be observed through the lens of static sample snapshots. While challenging, reconstructing their temporal evolution to decipher underlying dynamic properties is of major interest to scientific research. Existing approaches enable data transport along a temporal axis but are poorly scalable in high dimension and require restrictive assumptions to be met. To address these issues, we propose \textit{\textbf{Multi-Marginal temporal Schrödinger Bridge Matching}} (\textbf{MMtSBM}) \textit{for video generation from unpaired data}, extending the theoretical guarantees and empirical efficiency of Diffusion Schrödinger Bridge Matching (arXiv:archive/2303.16852) by deriving the Iterative Markovian Fitting algorithm to multiple marginals in a novel factorized fashion. Experiments show that MMtSBM retains theoretical properties on toy examples, achieves state-of-the-art performance on real world datasets such as transcriptomic trajectory inference in 100 dimensions, and for the first time recovers couplings and dynamics in very high dimensional image settings. Our work establishes multi-marginal Schrödinger bridges as a practical and principled approach for recovering hidden dynamics from static data.