Brenier Isotonic Regression
This work addresses the challenge of shape-constrained regression for multi-output data, which is incremental as it adapts existing isotonic regression concepts using optimal transport theory.
The paper tackles the problem of extending isotonic regression to multi-output settings by introducing cyclically monotone functions, which are gradients of convex potentials, and demonstrates that this approach, called Brenier isotonic regression, robustly outperforms many baselines in probability calibration tasks.
Isotonic regression (IR) is shape-constrained regression to maintain a univariate fitting curve non-decreasing, which has numerous applications including single-index models and probability calibration. When it comes to multi-output regression, the classical IR is no longer applicable because the monotonicity is not readily extendable. We consider a novel multi-output regression problem where a regression function is \emph{cyclically monotone}. Roughly speaking, a cyclically monotone function is the gradient of some convex potential. Whereas enforcing cyclic monotonicity is apparently challenging, we leverage the fact that Kantorovich's optimal transport (OT) always yields a cyclically monotone coupling as an optimal solution. This perspective naturally allows us to interpret a regression function and the convex potential as a link function in generalized linear models and Brenier's potential in OT, respectively, and hence we call this IR extension \emph{Brenier isotonic regression}. We demonstrate experiments with probability calibration and generalized linear models. In particular, IR outperforms many famous baselines in probability calibration robustly.