Geometric Losses for Distributional Learning
This work addresses the need for predicting sparse and singular distributions in machine learning, such as in ordinal regression and drawing generation, but appears incremental as it builds upon existing entropy-regularized optimal transport methods.
The paper tackles the problem of distributional learning by proposing a generalization of the logistic loss that incorporates a metric between classes, resulting in unconstrained convex objectives and supporting large class spaces. It demonstrates effectiveness in applications like ordinal regression and drawing generation, though no concrete numbers are provided in the abstract.
Building upon recent advances in entropy-regularized optimal transport, and upon Fenchel duality between measures and continuous functions , we propose a generalization of the logistic loss that incorporates a metric or cost between classes. Unlike previous attempts to use optimal transport distances for learning, our loss results in unconstrained convex objective functions, supports infinite (or very large) class spaces, and naturally defines a geometric generalization of the softmax operator. The geometric properties of this loss make it suitable for predicting sparse and singular distributions, for instance supported on curves or hyper-surfaces. We study the theoretical properties of our loss and show-case its effectiveness on two applications: ordinal regression and drawing generation.