Effective Parallelisation for Machine Learning
This addresses the open question of efficient parallelization in machine learning, particularly for critical applications, but is incremental as it builds on existing parallelization concepts.
The paper tackles the problem of efficiently parallelizing machine learning algorithms to handle increasing data and accuracy demands, presenting a scheme that reduces runtime to polylogarithmic time on many processors while maintaining theoretical guarantees, though at the cost of higher sample complexity.
We present a novel parallelisation scheme that simplifies the adaptation of learning algorithms to growing amounts of data as well as growing needs for accurate and confident predictions in critical applications. In contrast to other parallelisation techniques, it can be applied to a broad class of learning algorithms without further mathematical derivations and without writing dedicated code, while at the same time maintaining theoretical performance guarantees. Moreover, our parallelisation scheme is able to reduce the runtime of many learning algorithms to polylogarithmic time on quasi-polynomially many processing units. This is a significant step towards a general answer to an open question on the efficient parallelisation of machine learning algorithms in the sense of Nick's Class (NC). The cost of this parallelisation is in the form of a larger sample complexity. Our empirical study confirms the potential of our parallelisation scheme with fixed numbers of processors and instances in realistic application scenarios.