Forward-Backward Splitting for Optimal Transport based Problems
This work addresses the problem of efficient optimization for optimal transport extensions in machine learning, particularly for domain adaptation, but it is incremental as it builds on existing regularization and splitting techniques.
The authors tackled the challenge of designing efficient optimization algorithms for optimal transport-based extensions in machine learning by developing a general forward-backward splitting algorithm based on Bregman distances. The result showed a significant improvement in speed and performance for domain adaptation on a rotating distribution from the two moon dataset, compared to state-of-the-art methods.
Optimal transport aims to estimate a transportation plan that minimizes a displacement cost. This is realized by optimizing the scalar product between the sought plan and the given cost, over the space of doubly stochastic matrices. When the entropy regularization is added to the problem, the transportation plan can be efficiently computed with the Sinkhorn algorithm. Thanks to this breakthrough, optimal transport has been progressively extended to machine learning and statistical inference by introducing additional application-specific terms in the problem formulation. It is however challenging to design efficient optimization algorithms for optimal transport based extensions. To overcome this limitation, we devise a general forward-backward splitting algorithm based on Bregman distances for solving a wide range of optimization problems involving a differentiable function with Lipschitz-continuous gradient and a doubly stochastic constraint. We illustrate the efficiency of our approach in the context of continuous domain adaptation. Experiments show that the proposed method leads to a significant improvement in terms of speed and performance with respect to the state of the art for domain adaptation on a continually rotating distribution coming from the standard two moon dataset.