Methods for the Design and Evaluation of HCI+NLP Systems
This work addresses the challenge of integrating HCI's deep, small-scale evaluations with NLP's broad, benchmark-based methods for researchers and practitioners in both fields, but it is incremental as it builds on existing interdisciplinary ideas.
The paper tackles the differing evaluation methods between HCI and NLP, proposing five methodological approaches at their intersection to foster interdisciplinary collaboration and progress.
HCI and NLP traditionally focus on different evaluation methods. While HCI involves a small number of people directly and deeply, NLP traditionally relies on standardized benchmark evaluations that involve a larger number of people indirectly. We present five methodological proposals at the intersection of HCI and NLP and situate them in the context of ML-based NLP models. Our goal is to foster interdisciplinary collaboration and progress in both fields by emphasizing what the fields can learn from each other.