Distributionally Robust Logistic Regression
This work addresses robustness in logistic regression for scenarios with uncertain data distributions, offering a method that generalizes classical and regularized approaches, but it is incremental as it builds on existing distributionally robust optimization techniques.
The paper tackles the problem of logistic regression under distributional uncertainty by proposing a distributionally robust approach using Wasserstein distance to guarantee containment of the unknown data-generating distribution with high confidence, and it provides tractable reformulations and confidence bounds validated through experiments.
This paper proposes a distributionally robust approach to logistic regression. We use the Wasserstein distance to construct a ball in the space of probability distributions centered at the uniform distribution on the training samples. If the radius of this ball is chosen judiciously, we can guarantee that it contains the unknown data-generating distribution with high confidence. We then formulate a distributionally robust logistic regression model that minimizes a worst-case expected logloss function, where the worst case is taken over all distributions in the Wasserstein ball. We prove that this optimization problem admits a tractable reformulation and encapsulates the classical as well as the popular regularized logistic regression problems as special cases. We further propose a distributionally robust approach based on Wasserstein balls to compute upper and lower confidence bounds on the misclassification probability of the resulting classifier. These bounds are given by the optimal values of two highly tractable linear programs. We validate our theoretical out-of-sample guarantees through simulated and empirical experiments.