Learning Relevant Features of Data with Multi-scale Tensor Networks
This addresses efficient feature extraction for machine learning practitioners, though it appears incremental as it adapts existing physics methods to data.
The paper tackles the problem of feature learning for high-dimensional data by adapting coarse-graining approaches from physics to create multi-scale tensor network algorithms that scale linearly with input dimension and training set size. The result shows very good performance on MNIST and fashion-MNIST datasets, with a method that combines unsupervised representation with supervised optimization.
Inspired by coarse-graining approaches used in physics, we show how similar algorithms can be adapted for data. The resulting algorithms are based on layered tree tensor networks and scale linearly with both the dimension of the input and the training set size. Computing most of the layers with an unsupervised algorithm, then optimizing just the top layer for supervised classification of the MNIST and fashion-MNIST data sets gives very good results. We also discuss mixing a prior guess for supervised weights together with an unsupervised representation of the data, yielding a smaller number of features nevertheless able to give good performance.