Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysis
This provides theoretical insights for optimization in machine learning applications like generative modeling and domain adaptation, but it is incremental as it builds on existing SW methods.
The paper tackles the problem of understanding the gradient dynamics of the Sliced-Wasserstein Distance (SW) by analyzing its critical points, establishing that stable critical points cannot concentrate on segments through explicit perturbations and numerical experiments.
In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when employed as an objective functional. The SW metric has gained significant interest in the optimal transport and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining computationally tractable, making it a valuable tool for various applications, including generative modeling and domain adaptation. Our study aims to provide a rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concentrate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objective. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.