Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory
This work addresses the need for a mathematical and practical framework for SDRs in brain-inspired computing, but it appears incremental as it builds on existing HTM concepts without claiming major breakthroughs.
The paper tackles the problem of understanding and applying Sparse Distributed Representations (SDRs) as used in Hierarchical Temporal Memory (HTM) and the neocortex, deriving properties for scaling, robustness, and generalization to provide practical guidelines and a unified framework.
Empirical evidence demonstrates that every region of the neocortex represents information using sparse activity patterns. This paper examines Sparse Distributed Representations (SDRs), the primary information representation strategy in Hierarchical Temporal Memory (HTM) systems and the neocortex. We derive a number of properties that are core to scaling, robustness, and generalization. We use the theory to provide practical guidelines and illustrate the power of SDRs as the basis of HTM. Our goal is to help create a unified mathematical and practical framework for SDRs as it relates to cortical function.