Tensor networks and efficient descriptions of classical data
This work addresses the problem of scaling tensor network methods for machine learning applications, providing insights into their applicability to different data types, though it is incremental as it builds on existing quantum physics concepts.
The study investigated whether tensor networks can efficiently describe large image and text datasets by analyzing mutual information scaling, finding that text requires near-volume-law scaling (inefficient for 1D tensor networks) while images show near-area-law scaling (suitable for 2D tensor networks like PEPS).
We investigate the potential of tensor network based machine learning methods to scale to large image and text data sets. For that, we study how the mutual information between a subregion and its complement scales with the subsystem size $L$, similarly to how it is done in quantum many-body physics. We find that for text, the mutual information scales as a power law $L^ν$ with a close to volume law exponent, indicating that text cannot be efficiently described by 1D tensor networks. For images, the scaling is close to an area law, hinting at 2D tensor networks such as PEPS could have an adequate expressibility. For the numerical analysis, we introduce a mutual information estimator based on autoregressive networks, and we also use convolutional neural networks in a neural estimator method.