Compressing Deep Neural Networks: A New Hashing Pipeline Using Kac's Random Walk Matrices
This work addresses computational efficiency for deep learning practitioners, but it is incremental as it achieves similar rather than superior results.
The paper tackled the problem of compressing deep neural networks to reduce computational intensity by testing existing hashing pipelines and a new approach using Kac's random walk matrices, achieving similar accuracy to existing methods.
The popularity of deep learning is increasing by the day. However, despite the recent advancements in hardware, deep neural networks remain computationally intensive. Recent work has shown that by preserving the angular distance between vectors, random feature maps are able to reduce dimensionality without introducing bias to the estimator. We test a variety of established hashing pipelines as well as a new approach using Kac's random walk matrices. We demonstrate that this method achieves similar accuracy to existing pipelines.