Singular Value Decomposition and Neural Networks
arXiv:1906.11755v117 citations
Originality Incremental advance
AI Analysis
This provides a method for enhancing neural network training efficiency, though it appears incremental by applying linear algebra techniques to a known bottleneck.
The paper tackles the problem of improving neural network optimization by using Singular Value Decomposition (SVD) as a bridge to linear algebra and as an initial guess for parameters, resulting in substantially better optimization outcomes.
Singular Value Decomposition (SVD) constitutes a bridge between the linear algebra concepts and multi-layer neural networks---it is their linear analogy. Besides of this insight, it can be used as a good initial guess for the network parameters, leading to substantially better optimization results.