A Tunable Loss Function for Binary Classification
This work addresses the problem of designing flexible loss functions for binary classification in machine learning, offering an incremental improvement over existing methods.
The paper introduces α-loss, a tunable loss function for binary classification that bridges log-loss and 0-1 loss, proving it has desirable properties like classification calibration and an equivalent margin-based form, and shows that α=2 outperforms log-loss on MNIST with logistic regression.
We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal yet intractable $0$-$1$ loss. For logistic regression-based classification, we provide an upper bound on the difference between the empirical and expected risk at the empirical risk minimizers for $α$-loss by exploiting its Lipschitzianity along with recent results on the landscape features of empirical risk functions. Finally, we show that $α$-loss with $α= 2$ performs better than log-loss on MNIST for logistic regression.