PERSOMA: PERsonalized SOft ProMpt Adapter Architecture for Personalized Language Prompting
This addresses the challenge of building accurate and personalized natural language systems for users with evolving preferences, though it appears incremental as it builds upon existing embedding-based approaches.
The paper tackles the problem of efficiently capturing extensive user interaction histories for personalized language prompting by introducing PERSOMA, a novel architecture that resamples and compresses interactions into soft prompt embeddings, achieving superior performance in handling large and complex histories compared to existing methods.
Understanding the nuances of a user's extensive interaction history is key to building accurate and personalized natural language systems that can adapt to evolving user preferences. To address this, we introduce PERSOMA, Personalized Soft Prompt Adapter architecture. Unlike previous personalized prompting methods for large language models, PERSOMA offers a novel approach to efficiently capture user history. It achieves this by resampling and compressing interactions as free form text into expressive soft prompt embeddings, building upon recent research utilizing embedding representations as input for LLMs. We rigorously validate our approach by evaluating various adapter architectures, first-stage sampling strategies, parameter-efficient tuning techniques like LoRA, and other personalization methods. Our results demonstrate PERSOMA's superior ability to handle large and complex user histories compared to existing embedding-based and text-prompt-based techniques.