On Expected Accuracy
This work addresses the challenge of improving loss functions for classification in machine learning, but it appears incremental as it modifies an existing concept for specific optimization issues.
The authors tackled the problem of optimizing classification tasks by proposing a leaky version of expected accuracy as an alternative loss function to cross entropy, showing it yields comparable or better accuracy and is more robust to label noise across various tasks and architectures.
We empirically investigate the (negative) expected accuracy as an alternative loss function to cross entropy (negative log likelihood) for classification tasks. Coupled with softmax activation, it has small derivatives over most of its domain, and is therefore hard to optimize. A modified, leaky version is evaluated on a variety of classification tasks, including digit recognition, image classification, sequence tagging and tree tagging, using a variety of neural architectures such as logistic regression, multilayer perceptron, CNN, LSTM and Tree-LSTM. We show that it yields comparable or better accuracy compared to cross entropy. Furthermore, the proposed objective is shown to be more robust to label noise.