Large Language Model Augmented Exercise Retrieval for Personalized Language Learning
This work addresses the challenge of personalized exercise retrieval for online language learners, offering an incremental improvement over existing methods.
The paper tackles the problem of zero-shot exercise retrieval for personalized language learning by addressing the semantic gap between learner queries and exercise content, using large language models to synthesize hypothetical exercises for improved retrieval. The proposed mHyER method outperforms strong baselines on two novel benchmarks.
We study the problem of zero-shot exercise retrieval in the context of online language learning, to give learners the ability to explicitly request personalized exercises via natural language. Using real-world data collected from language learners, we observe that vector similarity approaches poorly capture the relationship between exercise content and the language that learners use to express what they want to learn. This semantic gap between queries and content dramatically reduces the effectiveness of general-purpose retrieval models pretrained on large scale information retrieval datasets like MS MARCO. We leverage the generative capabilities of large language models to bridge the gap by synthesizing hypothetical exercises based on the learner's input, which are then used to search for relevant exercises. Our approach, which we call mHyER, overcomes three challenges: (1) lack of relevance labels for training, (2) unrestricted learner input content, and (3) low semantic similarity between input and retrieval candidates. mHyER outperforms several strong baselines on two novel benchmarks created from crowdsourced data and publicly available data.