Support Vector Machine Active Learning Algorithms with Query-by-Committee versus Closest-to-Hyperplane Selection
This work addresses the problem of imbalanced datasets in active learning for applications like information extraction, but it is incremental as it builds on existing methods with specific modifications.
The paper tackled active learning for imbalanced datasets by comparing query-by-committee and closest-to-hyperplane selection combined with imbalance-handling methods, finding that the ClosestPA algorithm consistently outperformed others in text classification and relation extraction tasks.
This paper investigates and evaluates support vector machine active learning algorithms for use with imbalanced datasets, which commonly arise in many applications such as information extraction applications. Algorithms based on closest-to-hyperplane selection and query-by-committee selection are combined with methods for addressing imbalance such as positive amplification based on prevalence statistics from initial random samples. Three algorithms (ClosestPA, QBagPA, and QBoostPA) are presented and carefully evaluated on datasets for text classification and relation extraction. The ClosestPA algorithm is shown to consistently outperform the other two in a variety of ways and insights are provided as to why this is the case.