Mode-Aware Non-Linear Tucker Autoencoder for Tensor-based Unsupervised Learning
This work addresses computational and optimization issues in tensor-based unsupervised learning for researchers and practitioners dealing with high-dimensional data.
The paper tackles the challenge of high-dimensional tensor data in unsupervised learning by introducing the Mode-Aware Non-linear Tucker Autoencoder (MA-NTAE), which achieves linear computational complexity growth with tensor order and outperforms standard autoencoders and existing tensor networks in compression and clustering tasks, especially for higher-order tensors.
High-dimensional data, particularly in the form of high-order tensors, presents a major challenge in self-supervised learning. While MLP-based autoencoders (AE) are commonly employed, their dependence on flattening operations exacerbates the curse of dimensionality, leading to excessively large model sizes, high computational overhead, and challenging optimization for deep structural feature capture. Although existing tensor networks alleviate computational burdens through tensor decomposition techniques, most exhibit limited capability in learning non-linear relationships. To overcome these limitations, we introduce the Mode-Aware Non-linear Tucker Autoencoder (MA-NTAE). MA-NTAE generalized classical Tucker decomposition to a non-linear framework and employs a Pick-and-Unfold strategy, facilitating flexible per-mode encoding of high-order tensors via recursive unfold-encode-fold operations, effectively integrating tensor structural priors. Notably, MA-NTAE exhibits linear growth in computational complexity with tensor order and proportional growth with mode dimensions. Extensive experiments demonstrate MA-NTAE's performance advantages over standard AE and current tensor networks in compression and clustering tasks, which become increasingly pronounced for higher-order, higher-dimensional tensors.