Improving Existing Optimization Algorithms with LLMs
This addresses the problem of enhancing optimization efficiency for researchers and practitioners, but it is incremental as it builds on existing methods.
The paper tackled improving optimization algorithms by using LLMs to propose heuristic variations, showing that GPT-4o's alternative heuristic outperformed the expert-designed one in CMSA, with larger performance gaps on bigger and denser graphs.
The integration of Large Language Models (LLMs) into optimization has created a powerful synergy, opening exciting research opportunities. This paper investigates how LLMs can enhance existing optimization algorithms. Using their pre-trained knowledge, we demonstrate their ability to propose innovative heuristic variations and implementation strategies. To evaluate this, we applied a non-trivial optimization algorithm, Construct, Merge, Solve and Adapt (CMSA) -- a hybrid metaheuristic for combinatorial optimization problems that incorporates a heuristic in the solution construction phase. Our results show that an alternative heuristic proposed by GPT-4o outperforms the expert-designed heuristic of CMSA, with the performance gap widening on larger and denser graphs. Project URL: https://imp-opt-algo-llms.surge.sh/