Negative eigenvalues of the Hessian in deep neural networks
This work addresses the fundamental problem of non-convex optimization in deep learning for researchers, but it appears incremental as it builds on existing studies of Hessian properties.
The researchers investigated the role of negative eigenvalues in the Hessian matrix of deep neural networks to understand non-convexity in the loss landscape, but the abstract does not provide specific results or numbers.
The loss function of deep networks is known to be non-convex but the precise nature of this nonconvexity is still an active area of research. In this work, we study the loss landscape of deep networks through the eigendecompositions of their Hessian matrix. In particular, we examine how important the negative eigenvalues are and the benefits one can observe in handling them appropriately.