Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks
This provides a more efficient and accurate loss function for training deep neural networks on classification tasks, though it is incremental as it builds upon existing hinge loss concepts.
The paper tackles classification problems in deep networks by introducing the One-Sided Margin (OSM) loss function, achieving state-of-the-art accuracies such as 98.82% on CIFAR10 and faster training speeds compared to cross-entropy and hinge losses.
This paper introduces a new loss function, OSM (One-Sided Margin), to solve maximum-margin classification problems effectively. Unlike the hinge loss, in OSM the margin is explicitly determined with corresponding hyperparameters and then the classification problem is solved. In experiments, we observe that using OSM loss leads to faster training speeds and better accuracies than binary and categorical cross-entropy in several commonly used deep models for classification and optical character recognition problems. OSM has consistently shown better classification accuracies over cross-entropy and hinge losses for small to large neural networks. it has also led to a more efficient training procedure. We achieved state-of-the-art accuracies for small networks on several benchmark datasets of CIFAR10(98.82\%), CIFAR100(91.56\%), Flowers(98.04\%), Stanford Cars(93.91\%) with considerable improvements over other loss functions. Moreover, the accuracies are rather better than cross-entropy and hinge loss for large networks. Therefore, we strongly believe that OSM is a powerful alternative to hinge and cross-entropy losses to train deep neural networks on classification tasks.