Towards automated kernel selection in machine learning systems: A SYCL case study
This work addresses the need for more general kernel selection in machine learning libraries to accelerate research, though it is incremental as it builds on existing auto-tuning methods.
The paper tackles the problem of automated kernel selection for machine learning research, where network topologies and hyperparameters frequently change, by using machine learning to select high-performance SYCL kernels across heterogeneous devices, achieving high performance with minimal developer effort.
Automated tuning of compute kernels is a popular area of research, mainly focused on finding optimal kernel parameters for a problem with fixed input sizes. This approach is good for deploying machine learning models, where the network topology is constant, but machine learning research often involves changing network topologies and hyperparameters. Traditional kernel auto-tuning has limited impact in this case; a more general selection of kernels is required for libraries to accelerate machine learning research. In this paper we present initial results using machine learning to select kernels in a case study deploying high performance SYCL kernels in libraries that target a range of heterogeneous devices from desktop GPUs to embedded accelerators. The techniques investigated apply more generally and could similarly be integrated with other heterogeneous programming systems. By combining auto-tuning and machine learning these kernel selection processes can be deployed with little developer effort to achieve high performance on new hardware.