AssoMem: Scalable Memory QA with Multi-Signal Associative Retrieval
This work addresses the problem of scalable memory retrieval for AI assistants in conversational QA, offering a novel approach that improves recall accuracy in dense similarity contexts.
The paper tackles the challenge of accurate recall from large-scale memories for AI assistants in question answering, especially in similarity-dense scenarios, by proposing AssoMem, a framework that constructs an associative memory graph and integrates multi-dimensional retrieval signals, resulting in consistent outperformance of state-of-the-art baselines across three benchmarks and a new dataset.
Accurate recall from large scale memories remains a core challenge for memory augmented AI assistants performing question answering (QA), especially in similarity dense scenarios where existing methods mainly rely on semantic distance to the query for retrieval. Inspired by how humans link information associatively, we propose AssoMem, a novel framework constructing an associative memory graph that anchors dialogue utterances to automatically extracted clues. This structure provides a rich organizational view of the conversational context and facilitates importance aware ranking. Further, AssoMem integrates multi-dimensional retrieval signals-relevance, importance, and temporal alignment using an adaptive mutual information (MI) driven fusion strategy. Extensive experiments across three benchmarks and a newly introduced dataset, MeetingQA, demonstrate that AssoMem consistently outperforms SOTA baselines, verifying its superiority in context-aware memory recall.