Score-Based Model for Low-Rank Tensor Recovery
This work solves the challenge of multiway data analysis for applications like visual data processing, where predefined structural knowledge is often unavailable, though it appears incremental as it builds on existing probabilistic and optimization frameworks.
The paper tackles the problem of low-rank tensor recovery by addressing the limitations of traditional methods that rely on predefined structural assumptions, proposing a score-based model that learns compatibility between tensors and shared factors without such assumptions, and achieves significant performance improvements in tasks like tensor completion and denoising across various tensor types.
Low-rank tensor decompositions (TDs) provide an effective framework for multiway data analysis. Traditional TD methods rely on predefined structural assumptions, such as CP or Tucker decompositions. From a probabilistic perspective, these can be viewed as using Dirac delta distributions to model the relationships between shared factors and the low-rank tensor. However, such prior knowledge is rarely available in practical scenarios, particularly regarding the optimal rank structure and contraction rules. The optimization procedures based on fixed contraction rules are complex, and approximations made during these processes often lead to accuracy loss. To address this issue, we propose a score-based model that eliminates the need for predefined structural or distributional assumptions, enabling the learning of compatibility between tensors and shared factors. Specifically, a neural network is designed to learn the energy function, which is optimized via score matching to capture the gradient of the joint log-probability of tensor entries and shared factors. Our method allows for modeling structures and distributions beyond the Dirac delta assumption. Moreover, integrating the block coordinate descent (BCD) algorithm with the proposed smooth regularization enables the model to perform both tensor completion and denoising. Experimental results demonstrate significant performance improvements across various tensor types, including sparse and continuous-time tensors, as well as visual data.