MeGA: Merging Multiple Independently Trained Neural Networks Based on Genetic Algorithm
This provides a scalable solution for integrating pre-trained networks in deep learning applications, though it appears incremental as it builds on existing merging methods with a genetic optimization twist.
The paper tackles the problem of merging weights from multiple pre-trained neural networks by introducing MeGA, a method based on a genetic algorithm, which improves test accuracy on CIFAR-10 compared to individual models and conventional techniques like weight averaging.
In this paper, we introduce a novel method for merging the weights of multiple pre-trained neural networks using a genetic algorithm called MeGA. Traditional techniques, such as weight averaging and ensemble methods, often fail to fully harness the capabilities of pre-trained networks. Our approach leverages a genetic algorithm with tournament selection, crossover, and mutation to optimize weight combinations, creating a more effective fusion. This technique allows the merged model to inherit advantageous features from both parent models, resulting in enhanced accuracy and robustness. Through experiments on the CIFAR-10 dataset, we demonstrate that our genetic algorithm-based weight merging method improves test accuracy compared to individual models and conventional methods. This approach provides a scalable solution for integrating multiple pre-trained networks across various deep learning applications. Github is available at: https://github.com/YUNBLAK/MeGA-Merging-Multiple-Independently-Trained-Neural-Networks-Based-on-Genetic-Algorithm