TERM Model: Tensor Ring Mixture Model for Density Estimation
This work addresses interpretability and stability issues in density estimation for statistical machine learning, offering a novel ensemble-inspired approach that is incremental in improving upon existing tensor-based methods.
The paper tackles the challenge of efficient probability density estimation by proposing a tensor ring mixture model that reduces permutation candidates and enhances expressive capability, with experiments showing superiority in estimating density for moderately dimensional datasets and capturing intricate details in sampling.
Efficient probability density estimation is a core challenge in statistical machine learning. Tensor-based probabilistic graph methods address interpretability and stability concerns encountered in neural network approaches. However, a substantial number of potential tensor permutations can lead to a tensor network with the same structure but varying expressive capabilities. In this paper, we take tensor ring decomposition for density estimator, which significantly reduces the number of permutation candidates while enhancing expressive capability compared with existing used decompositions. Additionally, a mixture model that incorporates multiple permutation candidates with adaptive weights is further designed, resulting in increased expressive flexibility and comprehensiveness. Different from the prevailing directions of tensor network structure/permutation search, our approach provides a new viewpoint inspired by ensemble learning. This approach acknowledges that suboptimal permutations can offer distinctive information besides that of optimal permutations. Experiments show the superiority of the proposed approach in estimating probability density for moderately dimensional datasets and sampling to capture intricate details.