Jump Starting Bandits with LLM-Generated Prior Knowledge
This work addresses data efficiency in recommendation systems, but it is incremental as it applies an existing method (LLMs) to a new data generation task for bandits.
The paper tackles the problem of reducing online learning regret and data-gathering costs in contextual multi-armed bandits by integrating LLMs to generate prior knowledge, showing that this approach significantly cuts regret in experiments.
We present substantial evidence demonstrating the benefits of integrating Large Language Models (LLMs) with a Contextual Multi-Armed Bandit framework. Contextual bandits have been widely used in recommendation systems to generate personalized suggestions based on user-specific contexts. We show that LLMs, pre-trained on extensive corpora rich in human knowledge and preferences, can simulate human behaviours well enough to jump-start contextual multi-armed bandits to reduce online learning regret. We propose an initialization algorithm for contextual bandits by prompting LLMs to produce a pre-training dataset of approximate human preferences for the bandit. This significantly reduces online learning regret and data-gathering costs for training such models. Our approach is validated empirically through two sets of experiments with different bandit setups: one which utilizes LLMs to serve as an oracle and a real-world experiment utilizing data from a conjoint survey experiment.