MGE: A Training-Free and Efficient Model Generation and Enhancement Scheme
This provides an efficient method for building model pools, which is incremental but offers significant time savings for researchers and practitioners in deep learning.
The paper tackles the problem of constructing a model pool for deep learning by proposing MGE, a training-free scheme that generates models with comparable or superior performance to normally trained ones, using only 1% of the training time, and shows competitive generalization in few-shot tasks.
To provide a foundation for the research of deep learning models, the construction of model pool is an essential step. This paper proposes a Training-Free and Efficient Model Generation and Enhancement Scheme (MGE). This scheme primarily considers two aspects during the model generation process: the distribution of model parameters and model performance. Experiments result shows that generated models are comparable to models obtained through normal training, and even superior in some cases. Moreover, the time consumed in generating models accounts for only 1\% of the time required for normal model training. More importantly, with the enhancement of Evolution-MGE, generated models exhibits competitive generalization ability in few-shot tasks. And the behavioral dissimilarity of generated models has the potential of adversarial defense.