Empirical Study of Straggler Problem in Parameter Server on Iterative Convergent Distributed Machine Learning
This is an incremental study that addresses the straggler problem in distributed machine learning for researchers and developers working with parameter server architectures.
This study tested existing straggler mitigation techniques on iterative convergent machine learning algorithms like Matrix Factorization, Multinomial Logistic Regression, and Latent Dirichlet Allocation using the FlexPS system, finding that the results provide a platform for future research and comparison of methods.
The purpose of this study is to test the effectiveness of current straggler mitigation techniques over different important iterative convergent machine learning(ML) algorithm including Matrix Factorization (MF), Multinomial Logistic Regression (MLR), and Latent Dirichlet Allocation (LDA) . The experiment was conducted to implemented using the FlexPS system, which is the latest system implementation that employ parameter server architecture. The experiment employed the Bulk Synchronous Parallel (BSP) computational model to examine the straggler problem in Parameter Server on Iterative Convergent Distributed Machine Learning. Moreover, the current research analyzes the experimental arrangement of the parameter server strategy concerning the parallel learning problems by injecting universal straggler patterns and executing latest mitigation techniques. The findings of the study are significant in that as they will provide the necessary platform for conducting further research into the problem and allow the researcher to compare different methods for various applications. The outcome is therefore expected to facilitate the development of new techniques coupled with new perspectives in addressing this problem.