Efficient Sequential Decision Making with Large Language Models
This addresses the computational inefficiency of existing methods for integrating LLMs into decision-making processes, offering a more practical solution for AI applications.
The paper tackles the challenge of applying large language models (LLMs) to sequential decision making by proposing an online model selection approach, which achieves over a 6x performance gain on an Amazon dataset while using LLMs in only 1.5% of time steps.
This paper focuses on extending the success of large language models (LLMs) to sequential decision making. Existing efforts either (i) re-train or finetune LLMs for decision making, or (ii) design prompts for pretrained LLMs. The former approach suffers from the computational burden of gradient updates, and the latter approach does not show promising results. In this paper, we propose a new approach that leverages online model selection algorithms to efficiently incorporate LLMs agents into sequential decision making. Statistically, our approach significantly outperforms both traditional decision making algorithms and vanilla LLM agents. Computationally, our approach avoids the need for expensive gradient updates of LLMs, and throughout the decision making process, it requires only a small number of LLM calls. We conduct extensive experiments to verify the effectiveness of our proposed approach. As an example, on a large-scale Amazon dataset, our approach achieves more than a 6x performance gain over baselines while calling LLMs in only 1.5% of the time steps.