Efficient Probabilistic Tensor Networks
This work addresses computational bottlenecks for researchers and practitioners using PTNs in probabilistic modeling, offering incremental improvements in efficiency and scalability.
The paper tackles the problem of inefficient and unstable learning methods for probabilistic tensor networks (PTNs) by proposing a simple, numerically stable approach that achieves a 10x reduction in latency on MNIST and enables learning distributions with 10x more variables on density estimation benchmarks.
Tensor networks (TNs) enable compact representations of large tensors through shared parameters. Their use in probabilistic modeling is particularly appealing, as probabilistic tensor networks (PTNs) allow for tractable computation of marginals. However, existing approaches for learning parameters of PTNs are either computationally demanding and not fully compatible with automatic differentiation frameworks, or numerically unstable. In this work, we propose a conceptually simple approach for learning PTNs efficiently, that is numerically stable. We show our method provides significant improvements in time and space complexity, achieving 10x reduction in latency for generative modeling on the MNIST dataset. Furthermore, our approach enables learning of distributions with 10x more variables than previous approaches when applied to a variety of density estimation benchmarks. Our code is publicly available at github.com/marawangamal/ptn.