Rethinking Feature Distribution for Loss Functions in Image Classification
This work addresses the need for more robust loss functions in deep learning for image classification, offering incremental improvements over existing methods like softmax and its variants.
The paper tackles the problem of improving loss functions for image classification by proposing a large-margin Gaussian Mixture (L-GM) loss, which enhances classification performance and enables detection of abnormal inputs like adversarial examples, as demonstrated through experiments on benchmarks such as MNIST, CIFAR, ImageNet, and LFW.
We propose a large-margin Gaussian Mixture (L-GM) loss for deep neural networks in classification tasks. Different from the softmax cross-entropy loss, our proposal is established on the assumption that the deep features of the training set follow a Gaussian Mixture distribution. By involving a classification margin and a likelihood regularization, the L-GM loss facilitates both a high classification performance and an accurate modeling of the training feature distribution. As such, the L-GM loss is superior to the softmax loss and its major variants in the sense that besides classification, it can be readily used to distinguish abnormal inputs, such as the adversarial examples, based on their features' likelihood to the training feature distribution. Extensive experiments on various recognition benchmarks like MNIST, CIFAR, ImageNet and LFW, as well as on adversarial examples demonstrate the effectiveness of our proposal.