Tensor network language model
This work introduces a novel approach for machine learning in domains with complex correlations, though it appears incremental as it adapts existing tensor network methods from physics to language modeling.
The authors tackled the challenge of modeling systems with long-range correlations, such as natural language, by proposing a tensor network-based statistical model, and they provided analysis of its parameter space and applications like statistical translation.
We propose a new statistical model suitable for machine learning of systems with long distance correlations such as natural languages. The model is based on directed acyclic graph decorated by multi-linear tensor maps in the vertices and vector spaces in the edges, called tensor network. Such tensor networks have been previously employed for effective numerical computation of the renormalization group flow on the space of effective quantum field theories and lattice models of statistical mechanics. We provide explicit algebro-geometric analysis of the parameter moduli space for tree graphs, discuss model properties and applications such as statistical translation.