Hey Pentti, We Did (More of) It!: A Vector-Symbolic Lisp With Residue Arithmetic
This work addresses the need for more general intelligent agents by enabling neural networks to handle arbitrarily structured representations, though it appears incremental as an extension of existing methods.
The paper tackled the problem of encoding structured representations in neural networks by extending a Vector-Symbolic Architecture with residue arithmetic for Lisp 1.5, resulting in increased expressivity and inherent interpretability of network states.
Using Frequency-domain Holographic Reduced Representations (FHRRs), we extend a Vector-Symbolic Architecture (VSA) encoding of Lisp 1.5 with primitives for arithmetic operations using Residue Hyperdimensional Computing (RHC). Encoding a Turing-complete syntax over a high-dimensional vector space increases the expressivity of neural network states, enabling network states to contain arbitrarily structured representations that are inherently interpretable. We discuss the potential applications of the VSA encoding in machine learning tasks, as well as the importance of encoding structured representations and designing neural networks whose behavior is sensitive to the structure of their representations in virtue of attaining more general intelligent agents than exist at present.