Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search
This provides foundational insights for researchers in automated algorithm design, though it is incremental as it builds on existing LLM-assisted search frameworks.
The paper analyzes the fitness landscape of LLM-assisted algorithm search, revealing it is highly multimodal and rugged, especially in combinatorial optimization tasks, with structural variations across six tasks and six LLMs.
Using Large Language Models (LLMs) in an evolutionary or other iterative search framework have demonstrated significant potential in automated algorithm design. However, the underlying fitness landscape, which is critical for understanding its search behavior, remains underexplored. In this paper, we illustrate and analyze the fitness landscape of LLM-assisted Algorithm Search (LAS) using a graph-based approach, where nodes represent algorithms and edges denote transitions between them. We conduct extensive evaluations across six algorithm design tasks and six commonly-used LLMs. Our findings reveal that LAS landscapes are highly multimodal and rugged, particularly in combinatorial optimization tasks, with distinct structural variations across tasks and LLMs. Moreover, we adopt four different methods for algorithm similarity measurement and study their correlations to algorithm performance and operator behaviour. These insights not only deepen our understanding of LAS landscapes but also provide practical insights for designing more effective LAS methods.