LLMize: A Framework for Large Language Model-Based Numerical Optimization
This provides an accessible tool for practitioners in domains where optimization constraints are hard to formalize, though it is incremental as it adapts existing LLM capabilities to optimization.
The paper tackles the problem of applying large language models (LLMs) to numerical optimization by introducing LLMize, a framework that uses iterative prompting and in-context learning to handle black-box optimization with natural language constraints, and results show it is practical for complex domain-specific tasks but not competitive with classical solvers for simple problems.
Large language models (LLMs) have recently shown strong reasoning capabilities beyond traditional language tasks, motivating their use for numerical optimization. This paper presents LLMize, an open-source Python framework that enables LLM-driven optimization through iterative prompting and in-context learning. LLMize formulates optimization as a black-box process in which candidate solutions are generated in natural language, evaluated by an external objective function, and refined over successive iterations using solution-score feedback. The framework supports multiple optimization strategies, including Optimization by Prompting (OPRO) and hybrid LLM-based methods inspired by evolutionary algorithms and simulated annealing. A key advantage of LLMize is the ability to inject constraints, rules, and domain knowledge directly through natural language descriptions, allowing practitioners to define complex optimization problems without requiring expertise in mathematical programming or metaheuristic design. LLMize is evaluated on convex optimization, linear programming, the Traveling Salesman Problem, neural network hyperparameter tuning, and nuclear fuel lattice optimization. Results show that while LLM-based optimization is not competitive with classical solvers for simple problems, it provides a practical and accessible approach for complex, domain-specific tasks where constraints and heuristics are difficult to formalize.