Taking into Account the Differences between Actively and Passively Acquired Data: The Case of Active Learning with Support Vector Machines for Imbalanced Datasets
This addresses active learning challenges for imbalanced datasets in human language technology tasks, representing an incremental improvement over existing methods.
The paper tackles the problem of active learning with imbalanced datasets by proposing the InitPA method, which uses corpus-level imbalance estimates from small unbiased samples instead of labeled training data imbalance, achieving improved performance in active learning scenarios.
Actively sampled data can have very different characteristics than passively sampled data. Therefore, it's promising to investigate using different inference procedures during AL than are used during passive learning (PL). This general idea is explored in detail for the focused case of AL with cost-weighted SVMs for imbalanced data, a situation that arises for many HLT tasks. The key idea behind the proposed InitPA method for addressing imbalance is to base cost models during AL on an estimate of overall corpus imbalance computed via a small unbiased sample rather than the imbalance in the labeled training data, which is the leading method used during PL.