DARWIN: Dynamic Agentically Rewriting Self-Improving Network
This addresses the challenge of optimizing GPT training processes for researchers and practitioners, though it is incremental as it builds on existing evolutionary and GPT methods.
The paper tackled the problem of improving GPT model training efficiency and performance by introducing DARWIN, an evolutionary system where GPT agents iteratively modify each other's training code using genetic algorithms, resulting in a 1.26% improvement in model FLOPS utilization and a 2.07% improvement in perplexity over baseline configurations in 5 iterations.
DARWIN is an evolutionary GPT model, utilizing a genetic-algorithm like optimization structure with several independent GPT agents being trained individually using unique training code. Each iteration, the GPT models are prompted to modify the training code of one another in an attempt to improve their performance in a mutation-like manner, and the best GPT agents are then benchmarked and selected for the next iteration by genetic algorithm. For demonstration purposes and due to budget and time constraints, OpenAI API is used to prompt training code improvements and the nanoGPT framework is used as the training code. DARWIN also utilizes persistent JSON-based memory files to track previous reasoning and changes to code to correlate with improvement to model performance. and a bidirectional interface for HITL intervention allowing the model to request upgrades such as additional datasets, training scripts, and restructuring of file hierarchies. In experiments, DARWIN achieved a 1.26 percent improvement in model FLOPS utilization (MFU) and a 2.07 percent improvement to perplexity in 5 iterations of training over baseline configurations, demonstrating promising capabilities as a foundation for scaling evolutionary GPT training.