Lazy Parameter Tuning and Control: Choosing All Parameters Randomly From a Power-Law Distribution
This work addresses the problem of optimizing parameter settings for evolutionary algorithms, which is crucial for practitioners in optimization and AI, though it is incremental as it builds on existing methods with a novel twist.
The paper tackles the challenge of parameter tuning and control in evolutionary algorithms by proposing a lazy solution where all parameters are randomly chosen from a power-law distribution each iteration. The result shows that this approach achieves asymptotic runtimes comparable to or better than static parameters on problems like OneMax and jump functions, with proven performance guarantees and empirical confirmation.
Most evolutionary algorithms have multiple parameters and their values drastically affect the performance. Due to the often complicated interplay of the parameters, setting these values right for a particular problem (parameter tuning) is a challenging task. This task becomes even more complicated when the optimal parameter values change significantly during the run of the algorithm since then a dynamic parameter choice (parameter control) is necessary. In this work, we propose a lazy but effective solution, namely choosing all parameter values (where this makes sense) in each iteration randomly from a suitably scaled power-law distribution. To demonstrate the effectiveness of this approach, we perform runtime analyses of the $(1+(λ,λ))$ genetic algorithm with all three parameters chosen in this manner. We show that this algorithm on the one hand can imitate simple hill-climbers like the $(1+1)$ EA, giving the same asymptotic runtime on problems like OneMax, LeadingOnes, or Minimum Spanning Tree. On the other hand, this algorithm is also very efficient on jump functions, where the best static parameters are very different from those necessary to optimize simple problems. We prove a performance guarantee that is comparable to the best performance known for static parameters. For the most interesting case that the jump size $k$ is constant, we prove that our performance is asymptotically better than what can be obtained with any static parameter choice. We complement our theoretical results with a rigorous empirical study confirming what the asymptotic runtime results suggest.