Distribution-Free Pretraining of Classification Losses via Evolutionary Dynamics
For practitioners training classifiers, EDL offers a way to learn a loss function without real data, potentially reducing dependency on labeled datasets.
The paper introduces Evolutionary Dynamic Loss (EDL), a framework for pretraining a classification loss function using synthetic data, which can replace cross-entropy and achieve competitive or improved accuracy on CIFAR-10 with ResNet backbones.
We propose Evolutionary Dynamic Loss (EDL), a framework that learns a transferable classification loss in the probability space using unlimited synthetic prediction-label pairs, without accessing real samples during the main loss pretraining stage. EDL parameterizes the loss as a lightweight network and is trained with a semantics-free ranking-consistency objective that assigns larger penalties for more erroneous predictions. To robustly explore the space of loss functions, we optimize EDL via an evolutionary strategy and introduce chaotic mutation to improve exploration under noisy fitness evaluations. Experiments on CIFAR-10 with ResNet backbones show that EDL can serve as a drop-in replacement for cross-entropy and achieves competitive or improved accuracy, while ablation studies confirm that chaotic mutation yields faster convergence and better synthetic pretraining metrics than standard Gaussian mutation.