Useful Compact Representations for Data-Fitting
This work addresses computational efficiency for large-scale deterministic optimization problems, but it appears incremental as it builds on existing compact representation techniques.
The paper tackles the problem of dense Hessian matrices being prohibitive for large minimization problems without second derivative information by developing new compact representations parameterized by a choice of vectors, which reduce to existing formulas in special cases. The result demonstrates effectiveness for large eigenvalue computations, tensor factorizations, and nonlinear regressions, though no concrete numbers are provided.
For minimization problems without 2nd derivative information, methods that estimate Hessian matrices can be very effective. However, conventional techniques generate dense matrices that are prohibitive for large problems. Limited-memory compact representations express the dense arrays in terms of a low rank representation and have become the state-of-the-art for software implementations on large deterministic problems. We develop new compact representations that are parameterized by a choice of vectors and that reduce to existing well known formulas for special choices. We demonstrate effectiveness of the compact representations for large eigenvalue computations, tensor factorizations and nonlinear regressions.