EXACT: How to Train Your Accuracy
This addresses the issue of suboptimal results from surrogate losses in classification tasks for machine learning practitioners.
The paper tackles the problem of directly optimizing classification accuracy, which is discontinuous and not amenable to gradient-based methods, by introducing stochasticity to model outputs to optimize expected accuracy. Experiments on linear models and deep image classification show it is a powerful alternative to existing losses.
Classification tasks are usually evaluated in terms of accuracy. However, accuracy is discontinuous and cannot be directly optimized using gradient ascent. Popular methods minimize cross-entropy, hinge loss, or other surrogate losses, which can lead to suboptimal results. In this paper, we propose a new optimization framework by introducing stochasticity to a model's output and optimizing expected accuracy, i.e. accuracy of the stochastic model. Extensive experiments on linear models and deep image classification show that the proposed optimization method is a powerful alternative to widely used classification losses.