Mind the Gap: Aligning Knowledge Bases with User Needs to Enhance Mental Health Retrieval
This work addresses the challenge of building scalable and reliable mental health information repositories for users, though it is incremental as it applies existing retrieval-augmented generation methods with a novel targeting strategy.
The paper tackles the problem of mental health information retrieval systems performing poorly due to misalignment with user needs, by proposing a gap-informed framework for corpus augmentation that identifies underrepresented topics from naturalistic user data. The result shows that directed augmentation achieves near-optimal performance with modest expansions (e.g., 42% increase for Query Transformation to reach ~95% of reference corpus performance), significantly reducing content creation demands compared to non-directed methods.
Access to reliable mental health information is vital for early help-seeking, yet expanding knowledge bases is resource-intensive and often misaligned with user needs. This results in poor performance of retrieval systems when presented concerns are not covered or expressed in informal or contextualized language. We present an AI-based gap-informed framework for corpus augmentation that authentically identifies underrepresented topics (gaps) by overlaying naturalistic user data such as forum posts in order to prioritize expansions based on coverage and usefulness. In a case study, we compare Directed (gap-informed augmentations) with Non-Directed augmentation (random additions), evaluating the relevance and usefulness of retrieved information across four retrieval-augmented generation (RAG) pipelines. Directed augmentation achieved near-optimal performance with modest expansions--requiring only a 42% increase for Query Transformation, 74% for Reranking and Hierarchical, and 318% for Baseline--to reach ~95% of the performance of an exhaustive reference corpus. In contrast, Non-Directed augmentation required substantially larger and thus practically infeasible expansions to achieve comparable performance (232%, 318%, 403%, and 763%, respectively). These results show that strategically targeted corpus growth can reduce content creation demands while sustaining high retrieval and provision quality, offering a scalable approach for building trusted health information repositories and supporting generative AI applications in high-stakes domains.