MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation
This addresses the challenge of conversational consistency for users of chatbots, though it appears incremental as it builds on existing methods with tailored tuning.
The authors tackled the problem of maintaining consistency in long-range open-domain conversations by proposing MemoChat, a pipeline that tunes large language models to use self-composed memos, resulting in outperforming strong baselines in experiments across three testing scenarios.
We propose MemoChat, a pipeline for refining instructions that enables large language models (LLMs) to effectively employ self-composed memos for maintaining consistent long-range open-domain conversations. We demonstrate a long-range open-domain conversation through iterative "memorization-retrieval-response" cycles. This requires us to carefully design tailored tuning instructions for each distinct stage. The instructions are reconstructed from a collection of public datasets to teach the LLMs to memorize and retrieve past dialogues with structured memos, leading to enhanced consistency when participating in future conversations. We invite experts to manually annotate a test set designed to evaluate the consistency of long-range conversations questions. Experiments on three testing scenarios involving both open-source and API-accessible chatbots at scale verify the efficacy of MemoChat, which outperforms strong baselines. Our codes, data and models are available here: https://github.com/LuJunru/MemoChat.