A Short Note on Batch-efficient Divide-and-Conquer Algorithm for EigenDecomposition
It addresses the computational bottleneck of EigenDecomposition for mini-batches of matrices in deep neural networks, offering a speed improvement for larger matrices than previous work.
This paper proposes a batch-efficient Divide-and-Conquer algorithm for EigenDecomposition of matrices with dimensions up to 64, achieving faster computation than PyTorch's SVD function for mini-batches.
EigenDecomposition (ED) is at the heart of many computer vision algorithms and applications. One crucial bottleneck limiting its usage is the expensive computation cost, particularly for a mini-batch of matrices in deep neural networks. Our previous work proposed a dedicated QR-based ED algorithm for batched small matrices (dim${<}32$). This short paper targets the limitation and proposes a batch-efficient Divide-and-Conquer based ED algorithm for larger matrices. The numerical test shows that for a mini-batch of matrices whose dimensions are smaller than $64$, our method can be much faster than the Pytorch SVD function.