Intuitiveness in Active Teaching
This work addresses the problem of making machine learning algorithms easier to interact with for users who need to provide small amounts of training data, which is an incremental contribution to human-computer interaction in ML.
This paper investigates the intuitiveness of machine learning algorithms when actively taught by users. It introduces a theoretical framework for intuitiveness and an active teaching paradigm using a 2D spatial learning task. A large-scale user study with 800 participants interacting with two algorithms provides initial evidence that intuition significantly impacts human-machine interaction.
While Machine learning gives rise to astonishing results in automated systems, it is usually at the cost of large data requirements. This makes many successful algorithms from machine learning unsuitable for human-machine interaction, where the machine must learn from a small number of training samples that can be provided by a user within a reasonable time frame. Fortunately, the user can tailor the training data they create to be as useful as possible, severely limiting its necessary size -- as long as they know about the machine's requirements and limitations. Of course, acquiring this knowledge can in turn be cumbersome and costly. This raises the question of how easy machine learning algorithms are to interact with. In this work, we address this issue by analyzing the intuitiveness of certain algorithms when they are actively taught by users. After developing a theoretical framework of intuitiveness as a property of algorithms, we introduce an active teaching paradigm involving a prototypical two-dimensional spatial learning task as a method to judge the efficacy of human-machine interactions. Finally, we present and discuss the results of a large-scale user study into the performance and teaching strategies of 800 users interacting with two prominent machine learning algorithms in our system, providing first evidence for the role of intuition as an important factor impacting human-machine interaction.