Provably Sub-Linear Two-Timescale NeuroEvolution with Online Plasticity
For researchers in neuroevolution and reinforcement learning, this work offers theoretical grounding and empirical validation for combining evolution with online learning, though it is an incremental extension of existing ideas.
This paper provides the first regret analysis for a two-timescale neuroevolution framework (NEOL) that combines architecture search with online weight adaptation, proving sublinear regret. Empirically, NEOL achieves higher final fitness and lower variance than pure NEAT on four control benchmarks, competing with strong RL baselines.
NeuroEvolution of Augmenting Topologies (NEAT) is a widely used neuroevolution algorithm for learning neural network architectures and weights for control tasks. However, standard offline optimisation searches for connection strengths directly, which can scale poorly in high-dimensional weight spaces and more difficult continuous control problems. Hybrid methods that combine neuroevolution with online learning can address this challenge, but their theoretical properties remain underexplored. This paper gives the first regret analysis for a general NeuroEvolutionary Online Learning (NEOL) framework, which decouples learning into two timescales: an outer loop for architecture search and an inner loop for online weight adaptation via rewardmodulated plasticity. Under mild conditions, we prove that NEOL achieves sublinear regret. Empirically, under fixed interaction budgets on four standard control benchmarks, a NEAT-based NEOL implementation achieves higher final fitness and lower variance than pure NEAT, and is competitive with strong reinforcement learning (RL) baselines on several tasks. The results are supported byWilcoxon rank-sum tests and ablation studies. Overall, the findings show that online plasticity can improve the sample efficiency and robustness of two-timescale neuroevolution. Code is available at https://github.com/boobaa2001/NeuroEvolution Online Learning NEOL.