An Interactive Human-Machine Learning Interface for Collecting and Learning from Complex Annotations
This work addresses the need for more flexible annotation methods in machine learning, particularly for human annotators, though it is incremental as it builds on existing human-computer interaction concepts.
The paper tackled the problem of rigid annotation constraints in binary classification by introducing an interactive interface that allows annotators to use counterfactual examples alongside standard labels, resulting in improved model performance and user engagement.
Human-Computer Interaction has been shown to lead to improvements in machine learning systems by boosting model performance, accelerating learning and building user confidence. In this work, we aim to alleviate the expectation that human annotators adapt to the constraints imposed by traditional labels by allowing for extra flexibility in the form that supervision information is collected. For this, we propose a human-machine learning interface for binary classification tasks which enables human annotators to utilise counterfactual examples to complement standard binary labels as annotations for a dataset. Finally we discuss the challenges in future extensions of this work.