Tensor Products and Hyperdimensional Computing
This work provides foundational insights for researchers in vector symbolic architectures, though it appears incremental as it builds on prior graph embedding analysis.
The paper tackles the problem of understanding the mathematical foundations of hyperdimensional computing by establishing the tensor product as the central representation, showing it is the most general and expressive with properties like errorless unbinding and detection.
Following up on a previous analysis of graph embeddings, we generalize and expand some results to the general setting of vector symbolic architectures (VSA) and hyperdimensional computing (HDC). Importantly, we explore the mathematical relationship between superposition, orthogonality, and tensor product. We establish the tensor product representation as the central representation, with a suite of unique properties. These include it being the most general and expressive representation, as well as being the most compressed representation that has errorrless unbinding and detection.