EvoMerge: Neuroevolution for Large Language Models
This addresses a key limitation in AI training for improving model intelligence, though it appears incremental as it builds on existing techniques like model merging.
The paper tackles the problem of large language models losing reasoning ability during fine-tuning by introducing EvoMerge, a neuroevolution approach that uses model merging and fine-tuning to enhance models beyond conventional methods.
Extensive fine-tuning on Large Language Models does not always yield better results. Oftentimes, models tend to get better at imitating one form of data without gaining greater reasoning ability and may even end up losing some intelligence. Here I introduce EvoMerge, a systematic approach to large language model training and merging. Leveraging model merging for weight crossover and fine-tuning for weight mutation, EvoMerge establishes an evolutionary process aimed at pushing models beyond the limits of conventional fine-tuning.