Deep Active Learning in the Open World
This addresses the challenge of handling unfamiliar conditions in safety-critical AI applications, though it is incremental as it builds on existing active learning methods.
The paper tackles the problem of machine learning models performing poorly in open-world scenarios with out-of-distribution data by introducing ALOE, a novel active learning algorithm that accelerates class discovery and learning, outperforming traditional baselines on three long-tailed image classification benchmarks.
Machine learning models deployed in open-world scenarios often encounter unfamiliar conditions and perform poorly in unanticipated situations. As AI systems advance and find application in safety-critical domains, effectively handling out-of-distribution (OOD) data is crucial to building open-world learning systems. In this work, we introduce ALOE, a novel active learning algorithm for open-world environments designed to enhance model adaptation by incorporating new OOD classes via a two-stage approach. First, diversity sampling selects a representative set of examples, followed by energy-based OOD detection to prioritize likely unknown classes for annotation. This strategy accelerates class discovery and learning, even under constrained annotation budgets. Evaluations on three long-tailed image classification benchmarks demonstrate that ALOE outperforms traditional active learning baselines, effectively expanding known categories while balancing annotation cost. Our findings reveal a crucial tradeoff between enhancing known-class performance and discovering new classes, setting the stage for future advancements in open-world machine learning.