Generic construction of scale-invariantly coarse grained memory
This provides a foundational constraint for memory networks in biological or synthetic agents, addressing the need for efficient temporal information storage in real-world prediction tasks.
The paper tackles the problem of constructing a memory system that maximizes predictive information for natural signals with scale-free temporal fluctuations, showing that scale-invariant coarse-graining is achieved by encoding linear combinations of Laplace transforms and their inverses.
Encoding temporal information from the recent past as spatially distributed activations is essential in order for the entire recent past to be simultaneously accessible. Any biological or synthetic agent that relies on the past to predict/plan the future, would be endowed with such a spatially distributed temporal memory. Simplistically, we would expect that resource limitations would demand the memory system to store only the most useful information for future prediction. For natural signals in real world which show scale free temporal fluctuations, the predictive information encoded in memory is maximal if the past information is scale invariantly coarse grained. Here we examine the general mechanism to construct a scale invariantly coarse grained memory system. Remarkably, the generic construction is equivalent to encoding the linear combinations of Laplace transform of the past information and their approximated inverses. This reveals a fundamental construction constraint on memory networks that attempt to maximize predictive information storage relevant to the natural world.