SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective
This addresses a critical bottleneck in tensor network applications for computer vision and machine learning, offering a significant efficiency improvement over existing methods.
The paper tackles the computationally expensive tensor network structure search (TN-SS) problem by proposing SVDinsTN, a novel paradigm that eliminates repeated structure evaluations, achieving approximately 100 to 1000 times acceleration compared to state-of-the-art methods while maintaining comparable representation ability.
Tensor network (TN) representation is a powerful technique for computer vision and machine learning. TN structure search (TN-SS) aims to search for a customized structure to achieve a compact representation, which is a challenging NP-hard problem. Recent "sampling-evaluation"-based methods require sampling an extensive collection of structures and evaluating them one by one, resulting in prohibitively high computational costs. To address this issue, we propose a novel TN paradigm, named SVD-inspired TN decomposition (SVDinsTN), which allows us to efficiently solve the TN-SS problem from a regularized modeling perspective, eliminating the repeated structure evaluations. To be specific, by inserting a diagonal factor for each edge of the fully-connected TN, SVDinsTN allows us to calculate TN cores and diagonal factors simultaneously, with the factor sparsity revealing a compact TN structure. In theory, we prove a convergence guarantee for the proposed method. Experimental results demonstrate that the proposed method achieves approximately 100 to 1000 times acceleration compared to the state-of-the-art TN-SS methods while maintaining a comparable level of representation ability.