LARS-VSA: A Vector Symbolic Architecture For Learning with Abstract Rules
This work addresses the curse of compositionality in neuro-symbolic architectures, offering a more efficient solution for learning abstract rules from limited data.
The paper tackles the problem of interference in compositional learning by adapting a relational bottleneck strategy to hyperdimensional computing, resulting in a system that is significantly more efficient than state-of-the-art methods while maintaining higher or equal accuracy on various test datasets.
Human cognition excels at symbolic reasoning, deducing abstract rules from limited samples. This has been explained using symbolic and connectionist approaches, inspiring the development of a neuro-symbolic architecture that combines both paradigms. In parallel, recent studies have proposed the use of a "relational bottleneck" that separates object-level features from abstract rules, allowing learning from limited amounts of data . While powerful, it is vulnerable to the curse of compositionality meaning that object representations with similar features tend to interfere with each other. In this paper, we leverage hyperdimensional computing, which is inherently robust to such interference to build a compositional architecture. We adapt the "relational bottleneck" strategy to a high-dimensional space, incorporating explicit vector binding operations between symbols and relational representations. Additionally, we design a novel high-dimensional attention mechanism that leverages this relational representation. Our system benefits from the low overhead of operations in hyperdimensional space, making it significantly more efficient than the state of the art when evaluated on a variety of test datasets, while maintaining higher or equal accuracy.