Instance Selection Mechanisms for Human-in-the-Loop Systems in Few-Shot Learning
This work addresses cost reduction for researchers and practitioners adapting machine learning models to novel classes, but it is incremental as it builds on existing few-shot learning and HITL concepts.
The paper tackles the problem of high data labeling costs in few-shot learning by designing a human-in-the-loop system that uses mechanisms to acquire expert knowledge for uncertain predictions, showing it significantly accelerates model performance with negligible labeling effort.
Business analytics and machine learning have become essential success factors for various industries - with the downside of cost-intensive gathering and labeling of data. Few-shot learning addresses this challenge and reduces data gathering and labeling costs by learning novel classes with very few labeled data. In this paper, we design a human-in-the-loop (HITL) system for few-shot learning and analyze an extensive range of mechanisms that can be used to acquire human expert knowledge for instances that have an uncertain prediction outcome. We show that the acquisition of human expert knowledge significantly accelerates the few-shot model performance given a negligible labeling effort. We validate our findings in various experiments on a benchmark dataset in computer vision and real-world datasets. We further demonstrate the cost-effectiveness of HITL systems for few-shot learning. Overall, our work aims at supporting researchers and practitioners in effectively adapting machine learning models to novel classes at reduced costs.