Online Personalizing White-box LLMs Generation with Neural Bandits
This addresses the challenge of personalizing text generation for users in applications like news headlines, though it is incremental as it builds on existing neural bandit and LLM methods.
The study tackled the problem of efficiently personalizing content generation by large language models (LLMs) for individual users without creating separate models, by introducing an online method using neural bandit algorithms to optimize soft instruction embeddings based on user feedback, resulting in up to a 62.9% improvement in ROUGE scores and a 2.76% increase in LLM-agent evaluation for personalized news headline generation.
The advent of personalized content generation by LLMs presents a novel challenge: how to efficiently adapt text to meet individual preferences without the unsustainable demand of creating a unique model for each user. This study introduces an innovative online method that employs neural bandit algorithms to dynamically optimize soft instruction embeddings based on user feedback, enhancing the personalization of open-ended text generation by white-box LLMs. Through rigorous experimentation on various tasks, we demonstrate significant performance improvements over baseline strategies. NeuralTS, in particular, leads to substantial enhancements in personalized news headline generation, achieving up to a 62.9% improvement in terms of best ROUGE scores and up to 2.76% increase in LLM-agent evaluation against the baseline.