Para-active learning
This work addresses the challenge of training nonlinear models like kernel SVMs and neural networks more efficiently through parallelization, offering a practical solution for domains where such models are used.
The paper tackles the problem of parallelizing learning algorithms by proposing a generic strategy based on active learning principles, which leverages the parallelizable search for informative examples and shows that performance does not deteriorate with slightly outdated models, reporting preliminary experiments with kernel SVMs and SGD-trained neural networks.
Training examples are not all equally informative. Active learning strategies leverage this observation in order to massively reduce the number of examples that need to be labeled. We leverage the same observation to build a generic strategy for parallelizing learning algorithms. This strategy is effective because the search for informative examples is highly parallelizable and because we show that its performance does not deteriorate when the sifting process relies on a slightly outdated model. Parallel active learning is particularly attractive to train nonlinear models with non-linear representations because there are few practical parallel learning algorithms for such models. We report preliminary experiments using both kernel SVMs and SGD-trained neural networks.