CarMem: Enhancing Long-Term Memory in LLM Voice Assistants through Category-Bounding
This addresses personalization and privacy concerns in voice assistants, particularly for in-car applications, though it appears incremental as it builds on existing LLM-based methods.
The paper tackles the problem of voice assistants struggling to retain user preferences by proposing a long-term memory system structured around predefined categories, achieving F1-scores of 0.78 to 0.95 in preference extraction and reducing redundant preferences by 95%.
In today's assistant landscape, personalisation enhances interactions, fosters long-term relationships, and deepens engagement. However, many systems struggle with retaining user preferences, leading to repetitive user requests and disengagement. Furthermore, the unregulated and opaque extraction of user preferences in industry applications raises significant concerns about privacy and trust, especially in regions with stringent regulations like Europe. In response to these challenges, we propose a long-term memory system for voice assistants, structured around predefined categories. This approach leverages Large Language Models to efficiently extract, store, and retrieve preferences within these categories, ensuring both personalisation and transparency. We also introduce a synthetic multi-turn, multi-session conversation dataset (CarMem), grounded in real industry data, tailored to an in-car voice assistant setting. Benchmarked on the dataset, our system achieves an F1-score of .78 to .95 in preference extraction, depending on category granularity. Our maintenance strategy reduces redundant preferences by 95% and contradictory ones by 92%, while the accuracy of optimal retrieval is at .87. Collectively, the results demonstrate the system's suitability for industrial applications.