User Profile with Large Language Models: Construction, Updating, and Benchmarking
This work addresses the problem of building and updating accurate user profiles for personalized systems, but it is incremental as it applies existing LLMs to a specific domain with new datasets.
The paper tackles user profile modeling for personalized systems by introducing two open-source datasets for construction and updating, and presents a method using large language models (LLMs) like Mistral-7b and Llama2-7b to improve precision and recall in profile generation.
User profile modeling plays a key role in personalized systems, as it requires building accurate profiles and updating them with new information. In this paper, we present two high-quality open-source user profile datasets: one for profile construction and another for profile updating. These datasets offer a strong basis for evaluating user profile modeling techniques in dynamic settings. We also show a methodology that uses large language models (LLMs) to tackle both profile construction and updating. Our method uses a probabilistic framework to predict user profiles from input text, allowing for precise and context-aware profile generation. Our experiments demonstrate that models like Mistral-7b and Llama2-7b perform strongly in both tasks. LLMs improve the precision and recall of the generated profiles, and high evaluation scores confirm the effectiveness of our approach.