Revolutionizing Long-Term Memory in AI: New Horizons with High-Capacity and High-Speed Storage
This work addresses memory limitations for achieving artificial superintelligence, but it is incremental as it focuses on exploring existing ideas rather than introducing new methods.
The paper tackles the problem of information loss in AI long-term memory by critiquing the dominant 'extract then store' paradigm and advocating for alternative approaches like 'store then on-demand extract', with simple experiments confirming their effectiveness.
Driven by our mission of "uplifting the world with memory," this paper explores the design concept of "memory" that is essential for achieving artificial superintelligence (ASI). Rather than proposing novel methods, we focus on several alternative approaches whose potential benefits are widely imaginable, yet have remained largely unexplored. The currently dominant paradigm, which can be termed "extract then store," involves extracting information judged to be useful from experiences and saving only the extracted content. However, this approach inherently risks the loss of information, as some valuable knowledge particularly for different tasks may be discarded in the extraction process. In contrast, we emphasize the "store then on-demand extract" approach, which seeks to retain raw experiences and flexibly apply them to various tasks as needed, thus avoiding such information loss. In addition, we highlight two further approaches: discovering deeper insights from large collections of probabilistic experiences, and improving experience collection efficiency by sharing stored experiences. While these approaches seem intuitively effective, our simple experiments demonstrate that this is indeed the case. Finally, we discuss major challenges that have limited investigation into these promising directions and propose research topics to address them.