FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse
This addresses the efficiency bottleneck in long-horizon AI agents by enabling persistent cognition without redundant processing, though it appears incremental as it builds on existing latent memory concepts.
The paper tackles the problem of LLMs lacking dynamic context preservation by proposing FlashMem, a framework that distills intrinsic memory from reasoning states via computation reuse, achieving matching performance to heavy baselines while reducing inference latency by 5 times.
The stateless architecture of Large Language Models inherently lacks the mechanism to preserve dynamic context, compelling agents to redundantly reprocess history to maintain long-horizon autonomy. While latent memory offers a solution, current approaches are hindered by architectural segregation, relying on auxiliary encoders that decouple memory from the reasoning backbone. We propose FlashMem, a framework that distills intrinsic memory directly from transient reasoning states via computation reuse. Leveraging the property that internal representations uniquely encode input trajectories, FlashMem identifies the last hidden state as a sufficient statistic for the interaction history. This enables a Shared-KV Consolidator to synthesize memory by attending directly to the backbone's frozen cache, eliminating redundant re-parameterization. Furthermore, a parameter-free Cognitive Monitor leverages attention entropy to adaptively trigger consolidation only when high epistemic uncertainty is detected. Experiments demonstrate that FlashMem matches the performance of heavy baselines while reducing inference latency by 5 times, effectively bridging the gap between efficiency and persistent cognition.