Very Basics of Tensors with Graphical Notations: Unfolding, Calculations, and Decompositions
It provides foundational educational material for readers struggling with tensor basics, but it is incremental as it compiles existing knowledge without new research contributions.
This lecture note introduces tensor network diagrams as a graphical notation to simplify and intuitively represent complex tensor multiplications, covering essential operations like inner, outer, Hadamard, Kronecker, and Khatri-Rao products to aid understanding in signal processing and machine learning.
Tensor network diagram (graphical notation) is a useful tool that graphically represents multiplications between multiple tensors using nodes and edges. Using the graphical notation, complex multiplications between tensors can be described simply and intuitively, and it also helps to understand the essence of tensor products. In fact, most of matrix/tensor products including inner product, outer product, Hadamard product, Kronecker product, and Khatri-Rao product can be written in graphical notation. These matrix/tensor operations are essential building blocks for the use of matrix/tensor decompositions in signal processing and machine learning. The purpose of this lecture note is to learn the very basics of tensors and how to represent them in mathematical symbols and graphical notation. Many papers using tensors omit these detailed definitions and explanations, which can be difficult for the reader. I hope this note will be of help to such readers.