Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologisms
For researchers and designers of AI communication tools, this study reveals limitations of current AI in bridging the gap between non-native and native speaker writing, and highlights a mismatch between perceived and actual competence.
This paper evaluates AI tools for helping non-native speakers learn and use English neologisms in cross-cultural communication. A human-subjects study (N=234) found that AI Explanation of meaning and usage yielded the largest gains in native speaker-rated communicative competence compared to no support, while contextual appropriateness judgments showed no difference across conditions.
Neologisms and emerging slang are central to daily conversation, yet challenging for non-native speakers (NNS) to interpret and use appropriately in cross-cultural communication with native speakers (NS). NNS increasingly make use of Artificial Intelligence (AI) tools to learn these words. We study the utility of such tools in mediating an informal communication scenario through a human-subjects study (N=234): NNS participants learn English neologisms with AI support, write messages using the learned word to an NS friend, and judge contextual appropriateness of the neologism in two provided writing samples. Using both NS evaluator-rated communicative competence of NNS-produced writing and NNS' contextual appropriateness judgments, we compare three AI-based support conditions: AI Definition, AI Rewrite into simpler English, AI Explanation of meaning and usage, and Non-AI Dictionary for comparison. We show that AI Explanation yields the largest gains over no support in NS-rated competence, while contextual appropriateness judgments show indifference across support. NNS participants' self-reported perceptions tend to overestimate NS ratings, revealing a mismatch between perceived and actual competence. We further observe a significant gap between NNS- and NS-produced writing, highlighting the limitations of current AI tools and informing design for future tools.