Towards Better Spherical Sliced-Wasserstein Distance Learning with Data-Adaptive Discriminative Projection Direction
This work addresses a limitation in measuring discrepancies between spherical data distributions for applications in geology, medicine, computer vision, and deep learning, offering an incremental improvement over existing methods.
The paper tackles the problem that Spherical Sliced-Wasserstein (SSW) distance treats all projection directions equally, which is unrealistic for spherical data distributions, by proposing a data-adaptive Discriminative Spherical Sliced-Wasserstein (DSSW) distance that weights projection directions based on discriminative importance. The result shows improved performance over SSW and other state-of-the-art methods in tasks like gradient flows, density estimation, and self-supervised learning, with negligible or reduced computational overhead.
Spherical Sliced-Wasserstein (SSW) has recently been proposed to measure the discrepancy between spherical data distributions in various fields, such as geology, medical domains, computer vision, and deep representation learning. However, in the original SSW, all projection directions are treated equally, which is too idealistic and cannot accurately reflect the importance of different projection directions for various data distributions. To address this issue, we propose a novel data-adaptive Discriminative Spherical Sliced-Wasserstein (DSSW) distance, which utilizes a projected energy function to determine the discriminative projection direction for SSW. In our new DSSW, we introduce two types of projected energy functions to generate the weights for projection directions with complete theoretical guarantees. The first type employs a non-parametric deterministic function that transforms the projected Wasserstein distance into its corresponding weight in each projection direction. This improves the performance of the original SSW distance with negligible additional computational overhead. The second type utilizes a neural network-induced function that learns the projection direction weight through a parameterized neural network based on data projections. This further enhances the performance of the original SSW distance with less extra computational overhead. Finally, we evaluate the performance of our proposed DSSW by comparing it with several state-of-the-art methods across a variety of machine learning tasks, including gradient flows, density estimation on real earth data, and self-supervised learning.