Lower bounds for trace estimation via Block Krylov and other methods
This work provides theoretical insights into the computational limits of trace estimation methods, which is important for researchers in numerical linear algebra and machine learning, but it is incremental as it builds on existing Block Krylov and Hutchinson's method frameworks.
The paper tackles the problem of estimating the trace of matrix functions, such as tr(f(A)), by analyzing theoretical lower bounds on the number of queries needed, particularly for tr(W^{-p}) with Wishart matrices, and derives upper bounds on Krylov steps for functions like A^{-1/2} and A^{-1}.
This paper studies theoretical lower bounds for estimating the trace of a matrix function, $\text{tr}(f(A))$, focusing on methods that use Hutchinson's method along with Block Krylov techniques. These methods work by approximating matrix-vector products like $f(A)V$ using a Block Krylov subspace. This is closely related to approximating functions with polynomials. We derive theoretical upper bounds on how many Krylov steps are needed for functions such as $A^{-1/2}$ and $A^{-1}$ by analyzing the upper bounds from the polynomial approximation of their scalar equivalent. In addition, we also develop lower limits on the number of queries needed for trace estimation, specifically for $\text{tr}(W^{-p})$ where $W$ is a Wishart matrix. Our study clarifies the connection between the number of steps in Block Krylov methods and the degree of the polynomial used for approximation. This links the total cost of trace estimation to basic limits in polynomial approximation and how much information is needed for the computation.