Landing with the Score: Riemannian Optimization through Denoising
This addresses data-driven design problems central to modern generative AI, offering a novel method for optimization when standard manifold operations are unavailable, though it builds on existing diffusion model concepts.
The paper tackles the problem of Riemannian optimization over data manifolds given only implicitly through the data distribution, by introducing a link function that connects the data distribution to geometric operations, enabling recovery of essential manifold operations like retraction and Riemannian gradient computation. It proposes two efficient inference-time algorithms, Denoising Landing Flow and Denoising Riemannian Gradient Descent, with theoretical guarantees for feasibility and optimality, and demonstrates effectiveness on finite-horizon reference tracking tasks in data-driven control.
Under the data manifold hypothesis, high-dimensional data are concentrated near a low-dimensional manifold. We study the problem of Riemannian optimization over such manifolds when they are given only implicitly through the data distribution, and the standard manifold operations required by classical algorithms are unavailable. This formulation captures a broad class of data-driven design problems that are central to modern generative AI. Our key idea is to introduce a link function that connects the data distribution to the geometric operations needed for optimization. We show that this function enables the recovery of essential manifold operations, such as retraction and Riemannian gradient computation. Moreover, we establish a direct connection between our construction and the score function in diffusion models of the data distribution. This connection allows us to leverage well-studied parameterizations, efficient training procedures, and even pretrained score networks from the diffusion model literature to perform optimization. Building on this foundation, we propose two efficient inference-time algorithms -- Denoising Landing Flow (DLF) and Denoising Riemannian Gradient Descent (DRGD) -- and provide theoretical guarantees for both feasibility (approximate manifold adherence) and optimality (small Riemannian gradient norm). Finally, we demonstrate the effectiveness of our approach on finite-horizon reference tracking tasks in data-driven control, highlighting its potential for practical generative and design applications.