On periodic distributed representations using Fourier embeddings
For researchers working with periodic data in neural networks or cognitive modeling, this work provides a neurally-plausible representation scheme, though it is incremental as it builds on existing Fourier embedding and Spatial Semantic Pointer concepts.
The paper addresses the problem of processing periodic signals by proposing periodic distributed representations using Fourier embeddings, which avoid issues with angular measures and allow control over kernel shapes. It formalizes Dirichlet and periodic Gaussian kernels using Spatial Semantic Pointers.
Periodic signals are critical for representing physical and perceptual phenomena. Scalar, real angular measures, e.g., radians and degrees, result in difficulty processing and distinguishing nearby angles, especially when their absolute difference exceeds pi. We can avoid this problem by using real-valued, periodic embeddings in high-dimensional space. These representations also allow us to control the nature of their dot product similarities, allowing us to construct a variety of different kernel shapes. In this work, we aim of highlight how these representations can be constructed and focus on the formalization of Dirichlet and periodic Gaussian kernels using the neurally-plausible representation scheme of Spatial Semantic Pointers.