Evolving parametrized Loss for Image Classification Learning on Small Datasets
This addresses the challenge of training image classifiers with limited data, which is a common issue in domains like medical imaging or niche applications, though it appears incremental as it builds on existing meta-learning and evolutionary strategies.
The paper tackled the problem of improving generalization in image classification on small datasets by evolving a parametrized loss function using meta-learning, resulting in effective improvements over classical loss functions like cross-entropy and mean squared error.
This paper proposes a meta-learning approach to evolving a parametrized loss function, which is called Meta-Loss Network (MLN), for training the image classification learning on small datasets. In our approach, the MLN is embedded in the framework of classification learning as a differentiable objective function. The MLN is evolved with the Evolutionary Strategy algorithm (ES) to an optimized loss function, such that a classifier, which optimized to minimize this loss, will achieve a good generalization effect. A classifier learns on a small training dataset to minimize MLN with Stochastic Gradient Descent (SGD), and then the MLN is evolved with the precision of the small-dataset-updated classifier on a large validation dataset. In order to evaluate our approach, the MLN is trained with a large number of small sample learning tasks sampled from FashionMNIST and tested on validation tasks sampled from FashionMNIST and CIFAR10. Experiment results demonstrate that the MLN effectively improved generalization compared to classical cross-entropy error and mean squared error.