Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations
This work addresses the need for better language assessment tools for ESL learners, though it is incremental as it builds on existing evaluation methods.
The researchers tackled the problem of evaluating interactive dialogue quality for English as a Second Language speakers by developing a framework that collects interactivity labels and micro-level features, finding that certain features like reference words strongly correlate with interactivity quality.
We present an evaluation framework for interactive dialogue assessment in the context of English as a Second Language (ESL) speakers. Our framework collects dialogue-level interactivity labels (e.g., topic management; 4 labels in total) and micro-level span features (e.g., backchannels; 17 features in total). Given our annotated data, we study how the micro-level features influence the (higher level) interactivity quality of ESL dialogues by constructing various machine learning-based models. Our results demonstrate that certain micro-level features strongly correlate with interactivity quality, like reference word (e.g., she, her, he), revealing new insights about the interaction between higher-level dialogue quality and lower-level linguistic signals. Our framework also provides a means to assess ESL communication, which is useful for language assessment.