A Computational Model for Tensor Core Units
This work addresses the problem of algorithm design for domain-specific hardware like Tensor Cores, enabling broader exploitation of these systems for machine learning and computational tasks, though it is incremental as it builds on existing hardware models.
The authors tackled the need for efficient algorithms on tensor core hardware by proposing the TCU model to capture small matrix multiplication capabilities, resulting in fast algorithms for problems like matrix operations, graph algorithms, and DFT, with performance gains demonstrated through theoretical analysis.
To respond to the need of efficient training and inference of deep neural networks, a plethora of domain-specific hardware architectures have been introduced, such as Google Tensor Processing Units and NVIDIA Tensor Cores. A common feature of these architectures is a hardware circuit for efficiently computing a dense matrix multiplication of a given small size. In order to broaden the class of algorithms that exploit these systems, we propose a computational model, named the TCU model, that captures the ability to natively multiply small matrices. We then use the TCU model for designing fast algorithms for several problems, including matrix operations (dense and sparse multiplication, Gaussian Elimination), graph algorithms (transitive closure, all pairs shortest distances), Discrete Fourier Transform, stencil computations, integer multiplication, and polynomial evaluation. We finally highlight a relation between the TCU model and the external memory model.