Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture
This addresses the issue of fateful forgetting in neural networks for AI researchers, though it appears incremental as it builds on existing external memory techniques.
The paper tackles the problem of long-term forgetting in neural networks by introducing Memoria, a human-inspired memory architecture, and demonstrates its effectiveness in tasks like sorting, language modeling, and classification, surpassing conventional techniques.
Making neural networks remember over the long term has been a longstanding issue. Although several external memory techniques have been introduced, most focus on retaining recent information in the short term. Regardless of its importance, information tends to be fatefully forgotten over time. We present Memoria, a memory system for artificial neural networks, drawing inspiration from humans and applying various neuroscientific and psychological theories. The experimental results prove the effectiveness of Memoria in the diverse tasks of sorting, language modeling, and classification, surpassing conventional techniques. Engram analysis reveals that Memoria exhibits the primacy, recency, and temporal contiguity effects which are characteristics of human memory.