Sparse and Low-rank Tensor Estimation via Cubic Sketchings
This addresses tensor estimation for applications like high-order interaction pursuit in high-dimensional linear regression, representing an incremental advance with a novel method for a known bottleneck.
The paper tackles the problem of sparse and low-rank tensor estimation from cubic sketchings by developing a two-stage non-convex method based on sparse tensor decomposition and thresholded gradient descent, achieving exact recovery in noiseless cases and stable recovery in noisy cases with high probability, with the procedure shown to be rate-optimal under certain conditions.
In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy case with high probability. The non-asymptotic analysis sheds light on an interplay between optimization error and statistical error. The proposed procedure is shown to be rate-optimal under certain conditions. As a technical by-product, novel high-order concentration inequalities are derived for studying high-moment sub-Gaussian tensors. An interesting tensor formulation illustrates the potential application to high-order interaction pursuit in high-dimensional linear regression.