Input anticipating critical reservoirs show power law forgetting of unexpected input events
This addresses a fundamental limitation in reservoir computing for long-term memory applications, though it appears incremental as it builds on existing echo state network frameworks.
The paper investigates conditions under which echo state networks can exhibit power law forgetting instead of exponential memory decay, enabling traces of earlier events to persist for very long time spans. It achieves this through restricted recurrent connectivity and anticipation learning with input, demonstrating that power law forgetting can be realized in such networks.
Usually, reservoir computing shows an exponential memory decay. This paper investigates under which circumstances echo state networks can show a power law forgetting. That means traces of earlier events can be found in the reservoir for very long time spans. Such a setting requires critical connectivity exactly at the limit of what is permissible according the echo state condition. However, for general matrices the limit cannot be determined exactly from theory. In addition, the behavior of the network is strongly influenced by the input flow. Results are presented that use certain types of restricted recurrent connectivity and anticipation learning with regard to the input, where indeed power law forgetting can be achieved.