Visual Knowledge Tracing
This addresses the need to understand and model human learning processes in visual tasks, which is incremental as it applies existing methods to a new task.
The paper tackled the problem of tracing how human learners evolve their visual classification behavior during challenging tasks, and demonstrated that recurrent models can predict this behavior on medical image and species identification datasets.
Each year, thousands of people learn new visual categorization tasks -- radiologists learn to recognize tumors, birdwatchers learn to distinguish similar species, and crowd workers learn how to annotate valuable data for applications like autonomous driving. As humans learn, their brain updates the visual features it extracts and attend to, which ultimately informs their final classification decisions. In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks. We propose models that jointly extract the visual features used by learners as well as predicting the classification functions they utilize. We collect three challenging new datasets from real human learners in order to evaluate the performance of different visual knowledge tracing methods. Our results show that our recurrent models are able to predict the classification behavior of human learners on three challenging medical image and species identification tasks.