ML-Based Optimum Number of CUDA Streams for the GPU Implementation of the Tridiagonal Partition Method
For researchers optimizing GPU-based parallel algorithms, this provides a practical method to tune CUDA stream count, though it is incremental and domain-specific.
This work develops a heuristic to predict the optimal number of CUDA streams for GPU implementation of the tridiagonal partition method, using regression models for non-dominant GPU operations and stream creation overhead. The predicted optimum values match empirical data acceptably well.
This paper presents a heuristic for finding the optimum number of CUDA streams by using tools common to the modern AI-oriented approaches and applied to the parallel partition algorithm. A time complexity model for the GPU realization of the partition method is built. Further, a refined time complexity model for the partition algorithm being executed on multiple CUDA streams is formulated. Computational experiments for different SLAE sizes are conducted, and the optimum number of CUDA streams for each of them is found empirically. Based on the collected data a model for the sum of the times for the non-dominant GPU operations (that take part in the stream overlap) is formulated using regression analysis. A fitting non-linear model for the overhead time connected with the creation of CUDA streams is created. Statistical analysis is done for all the built models. An algorithm for finding the optimum number of CUDA streams is formulated. Using this algorithm, together with the two models mentioned above, predictions for the optimum number of CUDA streams are made. Comparing the predicted values with the actual data, the algorithm is deemed to be acceptably good.