Evolving Loss Functions for Specific Image Augmentation Techniques
This work addresses the need for tailored loss functions in computer vision to improve model performance with specific image augmentations, though it is incremental as it builds on prior Neural Loss Function Search research.
The paper tackled the problem of loss function performance varying with different image augmentation techniques, and found that an evolved loss function, the inverse bessel logarithm loss, outperformed cross-entropy in most experiments on datasets like CIFAR-10, CIFAR-100, CARS, Oxford-Flowers, and Caltech.
Previous work in Neural Loss Function Search (NLFS) has shown a lack of correlation between smaller surrogate functions and large convolutional neural networks with massive regularization. We expand upon this research by revealing another disparity that exists, correlation between different types of image augmentation techniques. We show that different loss functions can perform well on certain image augmentation techniques, while performing poorly on others. We exploit this disparity by performing an evolutionary search on five types of image augmentation techniques in the hopes of finding image augmentation specific loss functions. The best loss functions from each evolution were then taken and transferred to WideResNet-28-10 on CIFAR-10 and CIFAR-100 across each of the five image augmentation techniques. The best from that were then taken and evaluated by fine-tuning EfficientNetV2Small on the CARS, Oxford-Flowers, and Caltech datasets across each of the five image augmentation techniques. Multiple loss functions were found that outperformed cross-entropy across multiple experiments. In the end, we found a single loss function, which we called the inverse bessel logarithm loss, that was able to outperform cross-entropy across the majority of experiments.