A short letter on the dot product between rotated Fourier transforms
This work provides a foundational correction to the mathematical understanding of SSP similarity, which is incremental but important for cognitive modeling and deep learning applications involving spatial representations.
The paper tackled the problem of characterizing the similarity measure between Spatial Semantic Pointers (SSPs) used for representing continuous space, disproving a previous conjecture that it was Gaussian and deriving a trigonometric formula showing it is a product of sinc functions. This result establishes a direct mathematical link between spatial displacement and similarity, supporting neural network architectures for spatial structures.
Spatial Semantic Pointers (SSPs) have recently emerged as a powerful tool for representing and transforming continuous space, with numerous applications to cognitive modelling and deep learning. Fundamental to SSPs is the notion of "similarity" between vectors representing different points in $n$-dimensional space -- typically the dot product or cosine similarity between vectors with rotated unit-length complex coefficients in the Fourier domain. The similarity measure has previously been conjectured to be a Gaussian function of Euclidean distance. Contrary to this conjecture, we derive a simple trigonometric formula relating spatial displacement to similarity, and prove that, in the case where the Fourier coefficients are uniform i.i.d., the expected similarity is a product of normalized sinc functions: $\prod_{k=1}^{n} \operatorname{sinc} \left( a_k \right)$, where $\mathbf{a} \in \mathbb{R}^n$ is the spatial displacement between the two $n$-dimensional points. This establishes a direct link between space and the similarity of SSPs, which in turn helps bolster a useful mathematical framework for architecting neural networks that manipulate spatial structures.