Leveraging Large Language Models for Solving Rare MIP Challenges
This work addresses a domain-specific problem for users of mathematical optimization, but it is incremental as it builds on existing LLM fine-tuning methods.
The paper tackled the challenge of solving rare or specialized mixed integer programming (MIP) problems by proposing a recursively dynamic temperature method with chain-of-thought, resulting in better feasible solutions compared to other strategies and complementing traditional solvers like Gurobi by accelerating pruning and improving efficiency.
Mixed Integer Programming (MIP) has been extensively applied in areas requiring mathematical solvers to address complex instances within tight time constraints. However, as the problem scale increases, the complexity of model formulation and finding feasible solutions escalates significantly. In contrast, the model-building cost for end-to-end models, such as large language models (LLMs), remains largely unaffected by problem scale due to their pattern recognition capabilities. While LLMs, like GPT-4, without fine-tuning, can handle some traditional medium-scale MIP problems, they struggle with uncommon or highly specialized MIP scenarios. Fine-tuning LLMs can yield some feasible solutions for medium-scale MIP instances, but these models typically fail to explore diverse solutions when constrained by a low and constant temperature, limiting their performance. In this paper, we propose and evaluate a recursively dynamic temperature method integrated with a chain-of-thought approach. Our findings show that starting with a high temperature and gradually lowering it leads to better feasible solutions compared to other dynamic temperature strategies. Additionally, by comparing results generated by the LLM with those from Gurobi, we demonstrate that the LLM can produce solutions that complement traditional solvers by accelerating the pruning process and improving overall efficiency.