Enhancing Cross Entropy with a Linearly Adaptive Loss Function for Optimized Classification Performance
This is an incremental improvement for classification tasks using one-hot encoded labels.
The authors tackled the problem of classification optimization by proposing a linearly adaptive cross entropy loss function with an additional term based on predicted true class probability, which on CIFAR-100 with ResNet consistently outperformed standard cross entropy in accuracy while maintaining similar efficiency.
We propose the Linearly Adaptive Cross Entropy Loss function. This is a novel measure derived from the information theory. In comparison to the standard cross entropy loss function, the proposed one has an additional term that depends on the predicted probability of the true class. This feature serves to enhance the optimization process in classification tasks involving one-hot encoded class labels. The proposed one has been evaluated on a ResNet-based model using the CIFAR-100 dataset. Preliminary results show that the proposed one consistently outperforms the standard cross entropy loss function in terms of classification accuracy. Moreover, the proposed one maintains simplicity, achieving practically the same efficiency to the traditional cross entropy loss. These findings suggest that our approach could broaden the scope for future research into loss function design.