Efficient algorithms for the Hadamard decomposition
This work addresses matrix compression and data analysis challenges for researchers and practitioners, but it is incremental as it builds on existing Hadamard decomposition methods with algorithmic improvements.
The paper tackled the problem of efficiently computing the Hadamard decomposition for matrix compression by developing an algorithm using alternating optimization with SVD-inspired initialization and momentum updates, and extended it to support more than two matrices, showing effectiveness in experiments across diverse datasets.
The Hadamard decomposition is a powerful technique for data analysis and matrix compression, which decomposes a given matrix into the element-wise product of two or more low-rank matrices. In this paper, we develop an efficient algorithm to solve this problem, leveraging an alternating optimization approach that decomposes the global non-convex problem into a series of convex sub-problems. To improve performance, we explore advanced initialization strategies inspired by the singular value decomposition (SVD) and incorporate acceleration techniques by introducing momentum-based updates. Beyond optimizing the two-matrix case, we also extend the Hadamard decomposition framework to support more than two low-rank matrices, enabling approximations with higher effective ranks while preserving computational efficiency. Finally, we conduct extensive experiments to compare our method with the existing gradient descent-based approaches for the Hadamard decomposition and with traditional low-rank approximation techniques. The results highlight the effectiveness of our proposed method across diverse datasets.