Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distances
This work addresses a theoretical gap in machine learning for researchers using Sliced-Wasserstein distances, offering tools to improve discriminative power in applications like generative modeling, but it is incremental as it builds on existing PAC-Bayesian and SW frameworks.
The paper tackles the lack of statistical generalization guarantees for Sliced-Wasserstein distances beyond uniform slice distributions by leveraging PAC-Bayesian theory, providing generalization bounds for adaptive distributions, a method to learn optimal slice distributions, and empirical validation.
The Sliced-Wasserstein distance (SW) is a computationally efficient and theoretically grounded alternative to the Wasserstein distance. Yet, the literature on its statistical properties -- or, more accurately, its generalization properties -- with respect to the distribution of slices, beyond the uniform measure, is scarce. To bring new contributions to this line of research, we leverage the PAC-Bayesian theory and a central observation that SW may be interpreted as an average risk, the quantity PAC-Bayesian bounds have been designed to characterize. We provide three types of results: i) PAC-Bayesian generalization bounds that hold on what we refer as adaptive Sliced-Wasserstein distances, i.e. SW defined with respect to arbitrary distributions of slices (among which data-dependent distributions), ii) a principled procedure to learn the distribution of slices that yields maximally discriminative SW, by optimizing our theoretical bounds, and iii) empirical illustrations of our theoretical findings.