CueBuddy: helping non-native English speakers navigate English-centric STEM education
This addresses a specific issue for non-native English speakers in STEM education, particularly in the Global South, but is incremental as it builds on existing keyword spotting and translation technologies.
The paper tackles the problem of non-native English speakers struggling with technical jargon in STEM education by introducing CueBuddy, a system that provides real-time lexical cues through keyword spotting and multilingual glossary lookup to help students keep up with lectures without disruption.
Students across the world in STEM classes, especially in the Global South, fall behind their peers who are more fluent in English, despite being at par with them in terms of scientific prerequisites. While many of them are able to follow everyday English at ease, key terms in English stay challenging. In most cases, such students have had most of their course prerequisites in a lower resource language. Live speech translation to lower resource languages is a promising area of research, however, models for speech translation can be too expensive on a large scale and often struggle with technical content. In this paper, we describe CueBuddy, which aims to remediate these issues by providing real-time "lexical cues" through technical keyword spotting along real-time multilingual glossary lookup to help students stay up to speed with complex English jargon without disrupting their concentration on the lecture. We also describe the limitations and future extensions of our approach.